Second-order Markov Model Research Articles

Traffic Status Prediction Based on Multidimensional Feature Matching and 2nd-Order Hidden Markov Model (HMM)

Spatiotemporal data from urban road traffic are pivotal for intelligent transportation systems and urban planning. Nonetheless, missing data in traffic datasets is a common challenge due to equipment failures, communication issues, and monitoring limitations, especially the missing not at random (MNAR) problem. This research introduces an approach to address MNAR-type missing data in traffic status prediction, utilizing a multidimensional feature sequence and a second-order hidden Markov model (2nd-order HMM). First, this approach involves extracting spatiotemporal features for the preset data sections and spatial features for the sections to be predicted based on the traffic spatiotemporal characteristics. Second, using the extracted features, distinctive road traffic features are generated for each section. Furthermore, at specific intervals within the defined time period, nearest distance feature matching is introduced to ascertain the traffic attributes of the road section under prediction. Finally, relying on the matched status results, a 2nd-order HMM is employed to forecast the traffic status for subsequent moments within the defined time period. Experiments were carried out using datasets from Shenzhen City and compared against the hidden Markov models and contrast measure (HMM-C) method to affirm the efficacy of the proposed approach.

Enhancing vehicle localization by matching HD map with road marking detection

Accurate and robust vehicle localization, a fundamental task for autonomous driving, is still a challenging problem especially in GPS-denied scenarios. Recently, high definition (HD) map suggests a promising solution. However, the matching between online sensed data and HD map is difficult and time-consuming. In our work, road markings are selected as landmarks due to salient appearance features. Based on the detection results of road markings from the vehicle-borne images, the points of edge lines are employed to fit straight lines with RANSAC for outlier removal. The distances between the ego-vehicle and the fitted edge lines can be computed with camera-vehicle calibration in advance. Subsequently, the point-to-line distances from the sensed data are mapped into global linear constraints on the vehicle’s positions with the support of a lane-level HD map, which provides centimeter-level coordinates of road markings. Finally, the distances and the localization from the integrated navigation system (INS) are fused with the proposed linear Kalman filter based on the second-order Markov model (KF-MM2). The proposed method has been verified in different daily driving scenarios. Experimental results demonstrate that our method can achieve good performance with an average localization error of 0.53 m and a standard deviation of 0.17 m.

Novel Approaches for Resource Management Across Edge Servers

Edge computing aims at reducing computation and storage across the cloud and provides service with reduced latency. Edge devices can be mobile devices, routers, cameras, printers or any Internet of Things (IoT) devices that generate vast amounts of data. The processing of these data is done by virtual machines (VMs) present in the edge servers that are located within close proximity of the edge devices. This work proposes two models which predict resource contention at the edge servers, namely, a Dynamic Markov model for Resource Contention Prediction in Edge Cloud (DMRCP) and a Hybrid Cascade of Regression and Markov model for Resource Contention Prediction (CRMRCP). In DMRCP, a history matrix is updated based on the CPU utilization of a Virtual Machine (VM). This history matrix is used to update a transition probability matrix. This matrix is used to predict the future state of the VM. In the CRMRCP approach, the past CPU utilization values of the virtual machines in the edge servers are used for predicting a set of future CPU utilization values using linear regression, polynomial regression, lasso regression and ridge regression. Then, the predicted future CPU utilization values are used by the dynamic and the second-order Markov models to classify the state of the edge servers as overloaded, underloaded or normally loaded. In both the approaches, the VMs that may cause resource contention are predicted and are migrated to other edge servers such that the destination edge server does not get overloaded after the migration. The DMRCP method is compared with the first-order and the second-order Markov models and the number of VM migrations is analysed to evaluate the performance. The number of VM migrations in the CRMRCP method is compared with that in the second-order Markov model. The overall results prove that the number of VM migrations for the DMRCP is 52.9% less compared to the first-order Markov model and 21.1% less when compared to the second-order Markov model. The number of VM migrations in CRMRCP is reduced by 81.8% when ridge regression cascaded with the second-order Markov model is used.

SeqPolar: Sequence Matching of Polarized LiDAR Map With HMM for Intelligent Vehicle Localization

3D LiDAR maps are playing a more and more important role in intelligent vehicle localization. In this paper, the proposed SeqPolar localization method consists of two parts. The first part is a novel node-based LiDAR map representation method, termed polarized LiDAR map (PLM), and the second is a map matching algorithm based on a second-order hidden Markov model (HMM2) for vehicle localization. Unlike existing 3D LiDAR map, which is usually constructed by accumulating 3D LiDAR clouds collected at different times, the proposed polarized LiDAR map is generated from a set of nodes. Each node consists of three elements: a polarized LiDAR image, scene features extracted from the polarized LiDAR image, and sensor pose. The polarized LiDAR image encodes 3D coordinates and reflectivities of the 3D LiDAR cloud using a multi-channel image format, a more concise, straightforward, and structured representation of the 3D LiDAR cloud. In the localization step, we propose an HMM2-based method to match a sequence of input polarized images within the map nodes and find the nearest node from the PLM. Afterward, the vehicle is readily localized from the matched map node with 3D registration. The proposed SeqPolar localization method has been validated with the actual field dataset and the public KITTI database. Experimental results demonstrate that the proposed HMM2-based matching method can achieve up to 98% accuracy to find the nearest node from PLM. Moreover, the SeqPolar localization method based on the pre-built PLM can achieve 30-centimeter localization accuracy in average on both test datasets.

Markov Chain Probability Models to Describe Bi-modal Rainfall Pattern in Sri Lanka

Purpose: Sri Lanka is a tropical island with prominent four monsoonal seasons. The rainfall pattern of the country follows a bi-modal distribution with two peaks centred on two growing seasons. Model the bimodel rainfall distribution of Sri Lanka using the Markov probability chain model was the objective of this study. Research Method: Both first and second-order Markov probability models were developed for Anuradhapura, Badulla, Hambanthota and Katunayake using daily rainfall data for 1981–2011 period and the locations were selected to cover the major parts of the country. The model fit was done using Instat Statistical Programme. Findings: Both first and second-order Markov models successfully described bi-modal distribution of rainfall. In general, both transitional probabilities in the first order (p_rd and p_rr) and three transitional probabilities of second-order except rainfall after dry day and rainy day (p_rrd) followed a bi-modal pattern with two peaks. The sum of the logs of the rainfall amount (lr) and the amount of rainfall on rainy days (r_mean) also showed two peaks for two growing seasons. In both models, stations in the dry zone showed higher agreement in the simulated rainfall. Research Limitations: Lack of continuous long-term rainfall data is one of the major limitations. Originality/ Value: It is evident that both first and second-order Markov chain probability models are very much capable to explain the bi-modal rainfall distribution in Sri Lanka.

Evaluating the Efficiency of Off-Ball Screens in Elite Basketball Teams via Second-Order Markov Modelling

In basketball, the offensive movements on both strong and weak sides and tactical behavior play major roles in the effectiveness of a team’s offense. In the literature, studies are mostly focused on offensive actions, such as ball screens on the strong side. In the present paper, for the first time a second-order Markov model is defined to evaluate players’ interactions on the weak side, particularly for exploring the effectiveness of tactical structures and off-ball screens regarding the final outcome. The sample consisted of 1170 possessions of the FIBA Basketball Champions League 2018–2019. The variables of interest were the type of screen on the weak side, the finishing move, and the outcome of the shot. The model incorporates partial non-homogeneity according to the time of the execution (0–24″) and the quarter of playtime, and it is conditioned on the off-ball screen type. Regarding the overall performance, the results indicated that the outcome of each possession was influenced not only by the type of the executed shot, but also by the specific type of screen that took place earlier on the weak side of the offense. Thus, the proposed model could operate as an advisory tool for the coach’s strategic plans.

An Online Map Matching Algorithm Based on Second-Order Hidden Markov Model

Map matching is a key preprocess of trajectory data which recently have become a major data source for various transport applications and location-based services. In this paper, an online map matching algorithm based on the second-order hidden Markov model (HMM) is proposed for processing trajectory data in complex urban road networks such as parallel road segments and various road intersections. Several factors such as driver’s travel preference, network topology, road level, and vehicle heading are well considered. An extended Viterbi algorithm and a self-adaptive sliding window mechanism are adopted to solve the map matching problem efficiently. To demonstrate the effectiveness of the proposed algorithm, a case study is carried out using a massive taxi trajectory dataset in Nanjing, China. Case study results show that the accuracy of the proposed algorithm outperforms the baseline algorithm built on the first-order HMM in various testing experiments.

Identifying RNA N6-Methyladenine Sites in Three Species Based on a Markov Model

N6-methyladenosine (m6A), the most common posttranscriptional modification in eukaryotic mRNAs, plays an important role in mRNA splicing, editing, stability, degradation, etc. Since the methylation state is dynamic, methylation sequencing needs to be carried out over different time periods, which brings some difficulties to identify the RNA methyladenine sites. Thus, it is necessary to develop a fast and accurate method to identify the RNA N6-methyladenosine sites in the transcriptome. In this study, we use first-order and second-order Markov models to identify RNA N6-methyladenine sites in three species (Saccharomyces cerevisiae, mouse, and Homo sapiens). These two methods can fully consider the correlation between adjacent nucleotides. The results show that the performance of our method is better than that of other existing methods. Furthermore, the codons encoded by three nucleotides have biases in mRNA, and a second-order Markov model can capture this kind of information exactly. This may be the main reason why the performance of the second-order Markov model is better than that of the first-order Markov model in the m6A prediction problem. In addition, we provide a corresponding web tool called MM-m6APred.

Markov dependencies in the song syntax of Hermit Thrush (Catharus guttatus)

The sequences in which songbirds deliver their songs may help serve song’s primary functions related to territorial defense and mate attraction but determining that requires a clear understanding of the syntax of the species in question. Hermit Thrush (Catharus guttatus) males sing with a degree of predictability, cycling up and down the frequency spectrum with immediate variety. The present work used two complementary approaches to better understand Hermit Thrush song syntax by investigating the degree to which song type choice relies upon the identity of preceding songs within large (> 1000) song sets from 12 males. In one analysis, Fisher’s tests were used to assess whether the song immediately before the next song—or two, three, or four spots before it—influenced the next song. Results indicated that the next song is influenced by both the previous song and the one before it, and less so by those three or four songs back. In addition, simulated sequences were generated that reflect Markov models from zero-order (the next song depends on no preceding songs) through fourth-order (it depends on the previous four songs), and then compared back to the bird’s observed song sequence. This analysis revealed that naturally occurring sequences are best described by second-order Markov models, indicating a reliance upon the previous two songs. Together, the results indicate that Hermit Thrush song sequences are determined mainly by preceding two songs. Future work will further explore the presence of higher level dependencies, along with the role of syntax in territorial defense and aggression.

SOMM4mC: a second-order Markov model for DNA N4-methylcytosine site prediction in six species.

DNA N4-methylcytosine (4mC) modification is an important epigenetic modification in prokaryotic DNA due to its role in regulating DNA replication and protecting the host DNA against degradation. An efficient algorithm to identify 4mC sites is needed for downstream analyses. In this study, we propose a new prediction method named SOMM4mC based on a second-order Markov model, which makes use of the transition probability between adjacent nucleotides to identify 4mC sites. The results show that the first-order and second-order Markov model are superior to the three existing algorithms in all six species (Caenorhabditis elegans, Drosophila melanogaster, Arabidopsis thaliana, Escherichia coli, Geoalkalibacter subterruneus and Geobacter pickeringii) where benchmark datasets are available. However, the classification performance of SOMM4mC is more outstanding than that of first-order Markov model. Especially, for E.coli and C.elegans, the overall accuracy of SOMM4mC are 91.8% and 87.6%, which are 8.5% and 6.1% higher than those of the latest method 4mcPred-SVM, respectively. This shows that more discriminant sequence information is captured by SOMM4mC through the dependency between adjacent nucleotides. The web server of SOMM4mC is freely accessible at www.insect-genome.com/SOMM4mC. chenyuanyuan@njau.edu.cn or piancong@njau.edu.cn.

Predicting Long-Term Trajectories of Connected Vehicles via the Prefix-Projection Technique

The vehicle location prediction based on their spatial and temporal information is an important and difficult task in many applications. In the last few years, devices, such as connected vehicles, smart phones, GPS navigation systems, and smart home appliances, have amassed the large stores of geographic data. The task of leveraging this data by employing moving objects database techniques to predict spatio-temporal locations in an accurate and efficient fashion, comprising a complete trajectory remains an actively researched area. Existing methods for frequent sequential pattern mining tend to be limited to predicting short-term partial trajectories, at extremely high computational costs. In order to address these limitations, we designed a prefix-projection-based trajectory prediction algorithm called PrefixTP, which contains three essential phases. First, data collection, connected vehicles equipped with sensors comprise a vehicle grid and generate copious amounts of spatio-temporal data, in order to communicate and share traffic information. Second, model training, examining only the prefix subsequences, and projecting only their corresponding postfix subsequences into projected sets. Finally, trajectory matching, recursively finding postfix sequences meeting the requirement of minimum support count, and outputting the most frequent sequential pattern as the most probable trajectory. Fundamentally, PrefixTP supports three trajectory matching strategies which encompass all possibilities of prediction. Extensive experiments were conducted using real world GPS data sets, and the results show, when comparing predicted complete trajectories against partial short-term trajectories with a guarantee of real-time forecasting, that PrefixTP outperforms first-order, second-order Markov models, and Apriori-based trajectory prediction algorithm.

First and Second-Order Semi-Hidden Fritchman Markov models for a multi-carrier based indoor narrowband power line communication system

The realization of a Semi-Hidden Fritchman Markov models (SHFMMs) for power line communication (PLC) channel is only practicable if combined with efficient algorithms for learning and inference. This article thus reports First and Second-Order SHFMMs for the bit error pattern at the output of an Orthogonal Frequency Division Multiplexing based system for in-home narrowband PLC applications. Accurate SHFMMs have been derived for a residential and laboratory environment using the efficient iterative Baum–Welch algorithm. The log-likelihood ratio plots, the error-free run distribution plots, the mean square error and Chi-Square (χ2) test validates the accuracy of the derived models. The reliability and accuracy of resulting models are confirmed by a close match between the measured bit error sequences and the model generated bit error sequences. The estimated Second-Order SHFMMs have been validated to be superior to the First-Order models, although at the expense of more computational complexity. Resulting models assist designers to speed up and facilitate the process of designing and evaluating novelties for PLC systems.

Log-Viterbi algorithm applied on second-order hidden Markov model for human activity recognition

Recognition of human activities is getting into the limelight among researchers in the field of pervasive computing, ambient intelligence, robotic, and monitoring such as assistive living, elderly care, and health care. Many platforms, models, and algorithms have been developed and implemented to recognize the human activities. However, existing approaches suffer from low-activity accuracy and high time complexity. Therefore, we proposed probabilistic log-Viterbi algorithm on second-order hidden Markov model that facilitates our algorithm by reducing the time complexity with increased accuracy. Second-order hidden Markov model is efficient relevance between previous two activities, current activity, and current observation that incorporate more information into recognition procedure. The log-Viterbi algorithm converts the products of a large number of probabilities into additions and finds the most likely activity from observation sequence under given model. Therefore, this approach maximizes the probability of activity recognition with improved accuracy and reduced time complexity. We compared our proposed algorithm among other famous probabilistic models such as Naïve Bayes, condition random field, hidden Markov model, and hidden semi-Markov model using three datasets in the smart home environment. The recognition possibility of our proposed method is significantly better in accuracy and time complexity than early proposed method. Moreover, this improved algorithm for activity recognition is much effective for almost all the dynamic environments such as assistive living, elderly care, healthcare applications, and home automation.

Emirati-accented speaker identification in each of neutral and shouted talking environments

This work is devoted to capturing Emirati-accented speech database (Arabic United Arab Emirates database) in each of neutral and shouted talking environments in order to study and enhance text-independent Emirati-accented speaker identification performance in shouted environment based on each of First-Order Circular Suprasegmental Hidden Markov Models (CSPHMM1s), Second-Order Circular Suprasegmental Hidden Markov Models (CSPHMM2s), and Third-Order Circular Suprasegmental Hidden Markov Models (CSPHMM3s) as classifiers. In this research, our database was collected from fifty Emirati native speakers (twenty five per gender) uttering eight common Emirati sentences in each of neutral and shouted talking environments. The extracted features of our collected database are called Mel-Frequency Cepstral Coefficients (MFCCs). Our results show that average Emirati-accented speaker identification performance in neutral environment is 94.0%, 95.2%, and 95.9% based on CSPHMM1s, CSPHMM2s, and CSPHMM3s, respectively. On the other hand, the average performance in shouted environment is 51.3%, 55.5%, and 59.3% based, respectively, on CSPHMM1s, CSPHMM2s, and CSPHMM3s. The achieved average speaker identification performance in shouted environment based on CSPHMM3s is very similar to that obtained in subjective assessment by human listeners.

DMP-IOT: A distributed movement prediction scheme for IOT health-care applications

Mobility prediction in IP-based WSNs makes it possible to predict the next movement direction of mobile sensor nodes, which results in less power consumption and delay during handoff. The previous direction detection approaches need specific hardware facilities and impose considerable overhead during handoff. The advantage of DMP-IOT is distributing the learning model data around static sensors of the proposed tree, after the training phase. DMP-IOT includes a recovery mechanism that avoids disconnection of the mobile sensor node(s) to the network in case of a false prediction. The simulation results show about 25% improvement of DMP-IOT in saving power consumption and reducing handoff delay and packet loss, compared to movement direction approaches in similar works. The accuracy of the proposed movement prediction scheme is 83%, in average, which is validated by t−Student statistical test. Comparing the second-order Hidden Markov Model (HMM) with ANN reveals the superiority of the second-order HMM model in our application.

Sparse Trajectory Prediction Based on Multiple Entropy Measures

Trajectory prediction is an important problem that has a large number of applications. A common approach to trajectory prediction is based on historical trajectories. However, existing techniques suffer from the “data sparsity problem”. The available historical trajectories are far from enough to cover all possible query trajectories. We propose the sparsity trajectory prediction algorithm based on multiple entropy measures (STP-ME) to address the data sparsity problem. Firstly, the moving region is iteratively divided into a two-dimensional plane grid graph, and each trajectory is represented as a grid sequence with temporal information. Secondly, trajectory entropy is used to evaluate trajectory’s regularity, the L-Z entropy estimator is implemented to calculate trajectory entropy, and a new trajectory space is generated through trajectory synthesis. We define location entropy and time entropy to measure the popularity of locations and timeslots respectively. Finally, a second-order Markov model that contains a temporal dimension is adopted to perform sparse trajectory prediction. The experiments show that when trip completed percentage increases towards 90%, the coverage of the baseline algorithm decreases to almost 25%, while the STP-ME algorithm successfully copes with it as expected with only an unnoticeable drop in coverage, and can constantly answer almost 100% of query trajectories. It is found that the STP-ME algorithm improves the prediction accuracy generally by as much as 8%, 3%, and 4%, compared to the baseline algorithm, the second-order Markov model (2-MM), and sub-trajectory synthesis (SubSyn) algorithm, respectively. At the same time, the prediction time of STP-ME algorithm is negligible (10 μ s ), greatly outperforming the baseline algorithm (100 ms ).

Complexity, Predictability and Time Homogeneity of Syntax in the Songs of Cassin's Vireo (Vireo cassinii).

Many species of animals deliver vocalizations in sequences presumed to be governed by internal rules, though the nature and complexity of these syntactical rules have been investigated in relatively few species. Here I present an investigation into the song syntax of fourteen male Cassin’s Vireos (Vireo cassinii), a species whose song sequences are highly temporally structured. I compare their song sequences to three candidate models of varying levels of complexity–zero-order, first-order and second-order Markov models–and employ novel methods to interpolate between these three models. A variety of analyses, including sequence simulations, Fisher’s exact tests, and model likelihood analyses, showed that the songs of this species are too complex to be described by a zero-order or first-order Markov model. The model that best fit the data was intermediate in complexity between a first- and second-order model, though I also present evidence that some transition probabilities are conditioned on up to three preceding phrases. In addition, sequences were shown to be predictable with more than 54% accuracy overall, and predictability was positively correlated with the rate of song delivery. An assessment of the time homogeneity of syntax showed that transition probabilities between phrase types are largely stable over time, but that there was some evidence for modest changes in syntax within and between breeding seasons, a finding that I interpret to represent changes in breeding stage and social context rather than irreversible, secular shifts in syntax over time. These findings constitute a valuable addition to our understanding of bird song syntax in free-living birds, and will contribute to future attempts to understand the evolutionary importance of bird song syntax in avian communication.

Using higher-order Markov models to reveal flow-based communities in networks.

Complex systems made of interacting elements are commonly abstracted as networks, in which nodes are associated with dynamic state variables, whose evolution is driven by interactions mediated by the edges. Markov processes have been the prevailing paradigm to model such a network-based dynamics, for instance in the form of random walks or other types of diffusions. Despite the success of this modelling perspective for numerous applications, it represents an over-simplification of several real-world systems. Importantly, simple Markov models lack memory in their dynamics, an assumption often not realistic in practice. Here, we explore possibilities to enrich the system description by means of second-order Markov models, exploiting empirical pathway information. We focus on the problem of community detection and show that standard network algorithms can be generalized in order to extract novel temporal information about the system under investigation. We also apply our methodology to temporal networks, where we can uncover communities shaped by the temporal correlations in the system. Finally, we discuss relations of the framework of second order Markov processes and the recently proposed formalism of using non-backtracking matrices for community detection.

Speaker Identification in a Shouted Talking Environment Based on Novel Third-Order Circular Suprasegmental Hidden Markov Models

It is well known that speaker identification yields very high performance in a neutral talking environment, on the other hand, the performance has been sharply declined in a shouted talking environment. This work aims at proposing, implementing, and evaluating novel Third-Order Circular Suprasegmental Hidden Markov Models (CSPHMM3s) to improve the low performance of text-independent speaker identification in a shouted talking environment. CSPHMM3s possess combined characteristics of: Circular Hidden Markov Models (CHMMs), Third-Order Hidden Markov Models (HMM3s), and Suprasegmental Hidden Markov Models (SPHMMs). Our results show that CSPHMM3s are superior to each of: First-Order Left-to-Right Suprasegmental Hidden Markov Models (LTRSPHMM1s), Second-Order Left-to-Right Suprasegmental Hidden Markov Models (LTRSPHMM2s), Third-Order Left-to-Right Suprasegmental Hidden Markov Models (LTRSPHMM3s), First-Order Circular Suprasegmental Hidden Markov Models (CSPHMM1s), and Second-Order Circular Suprasegmental Hidden Markov Models (CSPHMM2s) in a shouted talking environment. Using our collected speech database, average speaker identification performance in a shouted talking environment based on LTRSPHMM1s, LTRSPHMM2s, LTRSPHMM3s, CSPHMM1s, CSPHMM2s, and CSPHMM3s is 74.6%, 78.4%, 81.7%, 78.7%, 83.4%, and 85.8%, respectively. Speaker identification performance that has been achieved based on CSPHMM3s is close to that attained based on subjective assessment by human listeners.

Regime Switching Modeling of Substance Use: Time-Varying and Second-Order Markov Models and Individual Probability Plots

A linear latent growth curve mixture model with regime switching is extended in 2 ways. Previously, the matrix of first-order Markov switching probabilities was specified to be time-invariant, regardless of the pair of occasions being considered. The first extension, time-varying transitions, specifies different Markov transition matrices between each pair of occasions. The second extension is second-order time-invariant Markov transition probabilities, such that the probability of switching depends on the states at the 2 previous occasions. The models are implemented using the R package OpenMx, which facilitates data handling, parallel computation, and further model development. It also enables the extraction and display of relative likelihoods for every individual in the sample. The models are illustrated with previously published data on alcohol use observed on 4 occasions as part of the National Longitudinal Survey of Youth, and demonstrate improved fit to the data.