Maximum Entropy Markov Model Research Articles

Velocity and Energy Consumption Prediction of Medium-Duty Electric Trucks Considering Road Features and Traffic Conditions

Abstract Electric vehicles (EVs) have emerged as a promising solution to address environmental concerns, especially benefiting urban delivery and last-mile fleets due to their unique operational characteristics. Despite the potential advantages, the adoption of electric trucks (eTrucks) into delivery fleets has been slow, mainly due to the challenge posed by eTrucks' limited driving range. Consequently, a reliable method for predicting the eTrucks' energy consumption in fleet route planning is essential, and the accuracy of the velocity trajectory forecast forming the fundamental basis. This paper introduces a data-driven approach to predict the velocity and energy consumption of medium-duty (MD) eTrucks, considering various road features, payload, and traffic conditions. A gated recurrent unit (GRU) is trained using traffic-labeled characteristic features specific to each road segment within a delivery route. For every predefined route, the GRU generates the velocity profile by analyzing a sequence of traffic states predicted from the maximum entropy Markov model (MEMM). Corresponding eTruck energy consumption is estimated using an autonomie truck model. Real-world EV data are used to evaluate the proposed method, and the results demonstrate that the model effectively utilizes the information, achieving high accuracy in predicting both eTruck velocity and energy consumption.

Part-of-Speech Tagging of Odia Language Using Statistical and Deep Learning Based Approaches

Automatic part-of-speech (POS) tagging is a preprocessing step of many natural language processing tasks, such as named entity recognition, speech processing, information extraction, word sense disambiguation, and machine translation. It has already gained promising results in English and European languages. However, in Indian languages, particularly in the Odia language, it is not yet well explored because of the lack of supporting tools, resources, and morphological richness of the language. Unfortunately, we were unable to locate an open source POS tagger for the Odia language, and only a handful of attempts have been made to develop POS taggers for the Odia language. The main contribution of this research work is to present statistical approaches such as the maximum entropy Markov model and conditional random field (CRF), as well as deep learning based approaches, including the convolutional neural network (CNN) and bidirectional long short-term memory (Bi-LSTM) to develop the Odia POS tagger. A publicly accessible corpus annotated with the Bureau of Indian Standards (BIS) tagset is used in our work. However, most of the languages around the globe have used the dataset annotated with the Universal Dependencies (UD) tagset. Hence, to maintain uniformity, the Odia dataset should use the same tagset. Thus, following the BIS and UD guidelines, we constructed a mapping from the BIS tagset to the UD tagset. The maximum entropy Markov model, CRF, Bi-LSTM, and CNN models are trained using the Indian Languages Corpora Initiative corpus with the BIS and UD tagsets. We have experimented with various feature sets as input to the statistical models to prepare a baseline system and observed the impact of constructed feature sets. The deep learning based model includes the Bi-LSTM network, the CNN network, the CRF layer, character sequence information, and a pre-trained word vector. Seven different combinations of neural sequence labeling models are implemented, and their performance measures are investigated. It has been observed that the Bi-LSTM model with the character sequence feature and pre-trained word vector achieved a result with 94.58% accuracy.

An Entropy-Based Method with a New Benchmark Dataset for Chinese Textual Affective Structure Analysis.

Affective understanding of language is an important research focus in artificial intelligence. The large-scale annotated datasets of Chinese textual affective structure (CTAS) are the foundation for subsequent higher-level analysis of documents. However, there are very few published datasets for CTAS. This paper introduces a new benchmark dataset for the task of CTAS to promote development in this research direction. Specifically, our benchmark is a CTAS dataset with the following advantages: (a) it is Weibo-based, which is the most popular Chinese social media platform used by the public to express their opinions; (b) it includes the most comprehensive affective structure labels at present; and (c) we propose a maximum entropy Markov model that incorporates neural network features and experimentally demonstrate that it outperforms the two baseline models.

Novel deep neural network framework for biomedical named entity recognition

With the dramatic improvements in the field of bio-informatics, extracting information from text and analyzing the association between the entities has received more attention in the past few years. Entity Recognition (ER) meant to extract and recognize the entities from any text. Biomedical Named Entity Recognition (BNER) gets more and more attention from the researchers since it is a fundamental task in biomedical information extraction. Various methods has been proposed to perform the task of BioNER. Different kind of approaches are dictionary based, rule based approaches, traditional machine learning approaches that combines supervised and unsupervised methods and neural network based approach. The state-of-art systems previously adopted various supervised machine learning methods Hidden Markov Models (HMMs), Maximum Entropy Markov Models (MEMMs), Support vector machines(SVM),Structural Support Vector Machines(SSVMS),Conditional Random Fields(CRF) to derive semantic and syntatic features from annotated datasets. However, CRF is one of the most successful method used for NER and it has obtained finest result because of the robustness and ability for sequence lebelling task. Recently, studies have demonstrated the application of deep learning based approaches for biomedical named entity recognition (BioNER) and shown promising results.

Extrinsic Behavior Prediction of Pedestrians via Maximum Entropy Markov Model and Graph-Based Features Mining

With the change of technology and innovation of the current era, retrieving data and data processing becomes a more challenging task for researchers. In particular, several types of sensors and cameras are used to collect multimedia data from various resources and domains, which have been used in different domains and platforms to analyze things such as educational and communicational setups, emergency services, and surveillance systems. In this paper, we propose a robust method to predict human behavior from indoor and outdoor crowd environments. While taking the crowd-based data as input, some preprocessing steps for noise reduction are performed. Then, human silhouettes are extracted that eventually help in the identification of human beings. After that, crowd analysis and crowd clustering are applied for more accurate and clear predictions. This step is followed by features extraction in which the deep flow, force interaction matrix and force flow features are extracted. Moreover, we applied the graph mining technique for data optimization, while the maximum entropy Markov model is applied for classification and predictions. The evaluation of the proposed system showed 87% of mean accuracy and 13% of error rate for the avenue dataset, while 89.50% of mean accuracy rate and 10.50% of error rate for the University of Minnesota (UMN) dataset. In addition, it showed a 90.50 mean accuracy rate and 9.50% of error rate for the A Day on Campus (ADOC) dataset. Therefore, these results showed a better accuracy rate and low error rate compared to state-of-the-art methods.

Indonesian part of speech tagging using maximum entropy markov model on Indonesian manually tagged corpus

This research discusses the development of a part of speech (POS) tagging system to solve the problem of word ambiguity. This paper presents a new method, namely maximum entropy markov model (MEMM) to solve word ambiguity on the Indonesian dataset. A manually labeled “Indonesian manually tagged corpus” was used as data. Furthermore, the corpus is processed using the entropy formula to obtain the weight of the value of the word being searched for, then calculating it into the MEMM Bigram and MEMM Trigram algorithms with the previously obtained rules to determine the part of speech (POS) tag that has the highest probability. The results obtained show POS tagging using the MEMM method has advantages over the methods used previously which used the same data. This paper improves a performance evaluation of research previously. The resulting average accuracy is 83.04% for the MEMM Bigram algorithm and 86.66% for the MEMM Trigram. The MEMM Trigram algorithm is better than the MEMM Bigram algorithm.

Automatic human posture estimation for sport activity recognition with robust body parts detection and entropy markov model

Automated human posture estimation (A-HPE) systems need delicate methods for detecting body parts and selecting cues based on marker-less sensors to effectively recognize complex activity motions. Recognition of human activities using vision sensors is a challenging issue due to variations in illumination conditions and complex movements during the monitoring of sports and fitness exercises. In this paper, we propose a novel A-HPE method that intelligently identifies human behaviours by utilizing saliency silhouette detection, robust body parts model and multidimensional cues from full-body silhouettes followed by an entropy Markov model. Initially, images are pre-processed and noise is removed to obtain a robust silhouette. Body parts models are then used to extract twelve key body parts. These key body parts are further optimized to assist the generation of multidimensional cues. These cues include energy, optical flow and distinctive values that are fed into quadratic discriminant analysis to discriminate cues which help in the recognition of actions. Finally, these optimized patterns are further processed by a maximum entropy Markov model as a recognizer engine based on transition and emission probability values for activity recognition. For evaluation, we used a leave-one-out cross validation scheme and the results outperformed existing well-known statistical state-of-the-art methods by achieving better body parts detection and higher recognition accuracy over four benchmark datasets. The proposed method will be useful for man-machine interactions such as 3D interactive games, virtual reality, service robots, e-health fitness, and security surveillance.

A Novel Chinese Entity Relationship Extraction Method Based on the Bidirectional Maximum Entropy Markov Model

To identify relationships among entities in natural language texts, extraction of entity relationships technically provides a fundamental support for knowledge graph, intelligent information retrieval, and semantic analysis, promotes the construction of knowledge bases, and improves efficiency of searching and semantic analysis. Traditional methods of relationship extraction, either those proposed at the earlier times or those based on traditional machine learning and deep learning, have focused on keeping relationships and entities in their own silos: extracting relationships and entities are conducted in steps before obtaining the mappings. To address this problem, a novel Chinese relationship extraction method is proposed in this paper. Firstly, the triple is treated as an entity relation chain and can identify the entity before the relationship and predict its corresponding relationship and the entity after the relationship. Secondly, the Joint Extraction of Entity Mentions and Relations model is based on the Bidirectional Long Short‐Term Memory and Maximum Entropy Markov Model (Bi‐MEMM). Experimental results indicate that the proposed model can achieve a precision of 79.2% which is much higher than that of traditional models.

Using machine learning techniques for rising star prediction in basketball

Rising stars in any field are the persons that have the potential to become popular in near future. Exploring rising stars in any organization will help the organization in their decisions making. Concept of rising star has been applied for finding rising authors in research community, rising business managers in telecommunication industry and rising players in the game of cricket. In this paper we presented the rising star prediction in basketball as a machine learning problem. We presented three types of co-players: co-players of same team in same game, co-players of opponent team in same game and co-players of both same and opponent team in same game. Co-players statistics are used as features for machine learning models. The co-player features are classified by feature size and type, which are further divided into different categories. Derived features along with their mathematical formulation are presented, that are derived from players statistics. The impact of co-players on prediction of rising star is measured through various machine learning models. Experimental results shows that derived features are dominant on different datasets in terms of F-measure score. The highest F-measure score achieved by derived features is 96%. Comparison of different machine learning models shows that Maximum Entropy Markov Model is dominant on all datasets in terms of F-measure score. The highest F-measure score achieved by Maximum Entropy Markov Model is 96%. Ranking comparison shows that most of the labeled rising stars are ranked in the top 100 in the subsequent six seasons. Comparison of rising stars with NBA (National Basketball Association) most improved players shows that rising stars have better efficiency in those seasons for which NBA most improved players were selected.

Accurate Physical Activity Recognition using Multidimensional Features and Markov Model for Smart Health Fitness

Recent developments in sensor technologies enable physical activity recognition (PAR) as an essential tool for smart health monitoring and for fitness exercises. For efficient PAR, model representation and training are significant factors contributing to the ultimate success of recognition systems because model representation and accurate detection of body parts and physical activities cannot be distinguished if the system is not well trained. This paper provides a unified framework that explores multidimensional features with the help of a fusion of body part models and quadratic discriminant analysis which uses these features for markerless human pose estimation. Multilevel features are extracted as displacement parameters to work as spatiotemporal properties. These properties represent the respective positions of the body parts with respect to time. Finally, these features are processed by a maximum entropy Markov model as a recognition engine based on transition and emission probability values. Experimental results demonstrate that the proposed model produces more accurate results compared to the state-of-the-art methods for both body part detection and for physical activity recognition. The accuracy of the proposed method for body part detection is 90.91% on a University of Central Florida’s (UCF) sports action dataset and, for activity recognition on a UCF YouTube action dataset and an IM-DailyRGBEvents dataset, accuracy is 89.09% and 88.26% respectively.

Artificial Intelligence in Operation Monitoring Discovers Patterns Within Drilling Reports

This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper OTC 29815, “Discovering Patterns Within Drilling Reports by Use of Artificial Intelligence for Operation Monitoring,” by Danilo Colombo, Petrobras, and Daniel Carlos Guimarães Pedronette and Ivan Rizzo Guilherme, UNESP, et al., prepared for the 2019 Offshore Technology Conference Brasil, Rio de Janeiro, 29-31 October. The paper has not been peer reviewed. Copyright 2020 Offshore Technology Conference. Reproduced by permission. In well-drilling activities, successful execution of a sequence of operations defined in a well project is critical. To provide proper monitoring, operations executed during drilling procedures are reported in daily drilling reports (DDRs). The complete paper provides an approach using machine-learning and sequence-mining algorithms for predicting and classifying the next operation based on textual descriptions. The general goal is to exploit the rich source of information represented by the DDRs to derive methodologies and tools capable of performing automatic data-analysis procedures and assisting human operators in time-consuming tasks. Methods Classification Tasks. fastText.This is a library discussed in the literature designed to learn word embeddings and text classification. The technique implements a simple linear model with rank constraint, and the text representation is a hidden state that is used to feed classifiers. A softmax function computes the probability distribution over pre-defined classes. Conditional Random Fields (CRFs). CRFs are a category of undirected graphical models that allow combination of features from each timestep of the sequence, with the ability to transit between labels for each episode in the input sequence. They were proposed to overcome the problem of bias that existed in techniques such as hidden Markov models and maximum-entropy Markov models. Recurrent Models. Despite achieving good results in several scenarios and learning word embeddings as a byproduct of its training, the fastText classifier does not properly consider word-ordering information that can be useful for several classification tasks. Such a shortcoming can be addressed by a recurrent neural network (RNN), which considers the fact that a fragment of text is formed by an ordered sequence of words. The authors consider the gated recurrent unit variant, which is easier to train than traditional RNNs and achieves results comparable with those of the long short-term memory unit, while figuring fewer parameters to learn. The methodology of these classifiers is detailed mathematically in the complete paper. Sequence Prediction. Sequential pattern mining can be defined broadly by the task of discovering interesting subsequences in a set of sequences, where the level of interest can be measured in terms of various criteria such as occurrence frequency, length, and profit, according to the application. The authors focus in this paper on the specific task of sequence prediction.

Automatic Recognition of Human Interaction via Hybrid Descriptors and Maximum Entropy Markov Model Using Depth Sensors.

Automatic identification of human interaction is a challenging task especially in dynamic environments with cluttered backgrounds from video sequences. Advancements in computer vision sensor technologies provide powerful effects in human interaction recognition (HIR) during routine daily life. In this paper, we propose a novel features extraction method which incorporates robust entropy optimization and an efficient Maximum Entropy Markov Model (MEMM) for HIR via multiple vision sensors. The main objectives of proposed methodology are: (1) to propose a hybrid of four novel features—i.e., spatio-temporal features, energy-based features, shape based angular and geometric features—and a motion-orthogonal histogram of oriented gradient (MO-HOG); (2) to encode hybrid feature descriptors using a codebook, a Gaussian mixture model (GMM) and fisher encoding; (3) to optimize the encoded feature using a cross entropy optimization function; (4) to apply a MEMM classification algorithm to examine empirical expectations and highest entropy, which measure pattern variances to achieve outperformed HIR accuracy results. Our system is tested over three well-known datasets: SBU Kinect interaction; UoL 3D social activity; UT-interaction datasets. Through wide experimentations, the proposed features extraction algorithm, along with cross entropy optimization, has achieved the average accuracy rate of 91.25% with SBU, 90.4% with UoL and 87.4% with UT-Interaction datasets. The proposed HIR system will be applicable to a wide variety of man–machine interfaces, such as public-place surveillance, future medical applications, virtual reality, fitness exercises and 3D interactive gaming.

Wearable Inertial Sensors for Daily Activity Analysis Based on Adam Optimization and the Maximum Entropy Markov Model.

Advancements in wearable sensors technologies provide prominent effects in the daily life activities of humans. These wearable sensors are gaining more awareness in healthcare for the elderly to ensure their independent living and to improve their comfort. In this paper, we present a human activity recognition model that acquires signal data from motion node sensors including inertial sensors, i.e., gyroscopes and accelerometers. First, the inertial data is processed via multiple filters such as Savitzky–Golay, median and hampel filters to examine lower/upper cutoff frequency behaviors. Second, it extracts a multifused model for statistical, wavelet and binary features to maximize the occurrence of optimal feature values. Then, adaptive moment estimation (Adam) and AdaDelta are introduced in a feature optimization phase to adopt learning rate patterns. These optimized patterns are further processed by the maximum entropy Markov model (MEMM) for empirical expectation and highest entropy, which measure signal variances for outperformed accuracy results. Our model was experimentally evaluated on University of Southern California Human Activity Dataset (USC-HAD) as a benchmark dataset and on an Intelligent Mediasporting behavior (IMSB), which is a new self-annotated sports dataset. For evaluation, we used the “leave-one-out” cross validation scheme and the results outperformed existing well-known statistical state-of-the-art methods by achieving an improved recognition accuracy of 91.25%, 93.66% and 90.91% when compared with USC-HAD, IMSB, and Mhealth datasets, respectively. The proposed system should be applicable to man–machine interface domains, such as health exercises, robot learning, interactive games and pattern-based surveillance.

A motifs-based Maximum Entropy Markov Model for realtime reliability prediction in System of Systems

System of Systems (SoS) based on service composition is considered as an effective way to build large-scale complex software systems. It regards the system as a service and integrates multiple component systems into a new system. The performance of the component system may fluctuate at any time because of the complex and changeable running state and external environment of the component system, which will affect the running of the SoS. The online reliability prediction technology is used to predict the reliability of the component system of an SoS in the near future. It aims to find errors and correct them in time so as to ensure that the SoS can run continuously and smoothly. To tackle the reliability prediction problem of component system in a dynamic and uncertain environment, the paper integrates Maximum Entropy Markov Model (MEMM) with time series motifs to achieve a new prediction model (m_MEMM), which is referred to as motifs-based MEMM. Extensive experiments are conducted to demonstrate the effectiveness and accuracy of the proposed approach.

A COMPARATIVE STUDY OF WORD REPRESENTATION METHODS WITH CONDITIONAL RANDOM FIELDS AND MAXIMUM ENTROPY MARKOV FOR BIO-NAMED ENTITY RECOGNITION

Bio-Named Entity Recognition (Bio-NER) is the process of identifying and semantically classifying biomedical technical terms and named entities in Biomedicine literature. Therefore, it is a major task in biomedical knowledge acquisition. Meanwhile, Natural Language Processing (NLP) plays an important role in Bio-NER in the biomedical domain. The first and most essential biomedical literature mining task incorporates biomedical entity recognition such as protein, gene, and chemicals. The most recent Bio-NER methods rely on predefined traditional features, which attempt to capture the specific surface properties of entity types. However, these empirically predefined feature sets differ between entity types and are manually constructed and complicated, which means developing them is costly. In this paper, we systematically present a comparative evaluation study of three methods, which are: the traditional feature representation method, the continuous bag-of-words (CBOW) model, and a new prototypical representation method with two popular sequence-labeling approaches (Conditional Random Fields (CRFs) and Maximum Entropy Markov Models (MEMM)). We evaluated these models with two major Bio-NER tasks, which involve the JNLPBA and GENETAG corpora. This paper examined the prototypical word representation method and found that Word2Vec can be successfully used for Bio-NER. Our results show that the new prototypical representation method improved the performance of the two machine learning models with different datasets. Also, the new prototypical representation method performed better than the traditional feature representation method and CBOW model for both datasets. Finally, our experiment proved that the CRF classifier with the new prototypical representation method achieved the best results when 90% data was used as training data, yielding overall F-measure values of 0.79% and 0.85% for the JNLPBA corpus and GENETAG corpus, respectively. In comparison, the results achieved using the ME classifier yielded overall F-measure values of 0.76% and 0.78% for the JNLPBA corpus and GENETAG corpus, respectively.

Identifying bacterial biotope entities using sequence labeling: Performance and feature analysis

Habitat information is important to biodiversity conservation and research. Extracting bacterial biotope entities from scientific publications is important to large scale study of the relationships between bacteria and their living environments. To facilitate the further development of robust habitat text mining systems for biodiversity, following the BioNLP task framework, three sequence labeling techniques, CRFs (Conditional Random Fields), MEMM (Maximum Entropy Markov Model) and SVMhmm (Support Vector Machine) and one classifier, SVMmulticlass, are compared on their performance in identifying three types of bacterial biotope entities: bacteria, habitats and geographical locations. The effectiveness of a variety of basic word formation features, syntactic features, and semantic features are exploited and compared for the three sequence labeling methods. Experiments on two publicly available BioNLP collections show that, in addition to a WordNet feature, word embedding featured clusters (although not trained with the task‐specific corpus) consistently improve the performance for all methods on all entity types in both collections. Other features produce various results. Our results also show that when trained on limited corpora, Brown clusters resulted in better performance than word embedding clusters did. Further analysis suggests that the entity recognition performance can be greatly boosted through improving the accuracy of entity boundary identification.

Named Entity Recognition on Code-Mixed Cross-Script Social Media Content

Focusing on the current multilingual scenario in social media, this paper reports automatic extraction of named entities (NE) from code-mixed cross-script social media data. Our prime target is to extract NE for question answering. This paper also introduces a Bengali-English (Bn-En) code-mixedcross-script dataset for NE research and proposes domain specific taxonomies for NE. We used formal as well as informal language-specific features to prepare the classification models and employed four machine learning algorithms (Conditional Random Fields, Margin Infused Relaxed Algorithm, Support Vector Machine and Maximum Entropy Markov Model) for the NE recognition (NER) task. In this study, Bengali is considered as the native language while English is considered as the non-native language. However, the approach presented in this paper is generic in nature and could be used for any other code-mixed dataset. The classification models based on CRF and SVM performed well among the classifiers.

EM-Psychiatry: An Ambient Intelligent System for Psychiatric Emergency

The proliferation of the market in patient care services is attracting attention in the healthcare industry; however, a remote mental healthcare system is still unattainable. In this paper, an ambient intelligent system of in-home psychiatric care service for emergency psychiatry (EM-psychiatry) is proposed for the remote monitoring of psychiatric emergency patients. The emergency psychiatric states of patients are modeled as the states of the maximum-entropy Markov model (MEMM), in which sensor observations, psychiatric screening scores, and patients’ histories are considered as the observations of MEMM. A modified Viterbi, a machine-learning algorithm, is used to generate the most probable psychiatric state sequence based on such observations; then, from the most likely psychiatric state sequence, the emergency psychiatric state is predicted through the proposed algorithm. The ambient EM-psychiatry model is implemented and the performance of the proposed prediction model is analyzed using the receiver operator characteristics curves, which demonstrates that the use of the EM-psychiatric screening questionnaire with biosensor observations enhances the prediction accuracy.

A Novel Maximum Entropy Markov Model for Human Facial Expression Recognition.

Research in video based FER systems has exploded in the past decade. However, most of the previous methods work well when they are trained and tested on the same dataset. Illumination settings, image resolution, camera angle, and physical characteristics of the people differ from one dataset to another. Considering a single dataset keeps the variance, which results from differences, to a minimum. Having a robust FER system, which can work across several datasets, is thus highly desirable. The aim of this work is to design, implement, and validate such a system using different datasets. In this regard, the major contribution is made at the recognition module which uses the maximum entropy Markov model (MEMM) for expression recognition. In this model, the states of the human expressions are modeled as the states of an MEMM, by considering the video-sensor observations as the observations of MEMM. A modified Viterbi is utilized to generate the most probable expression state sequence based on such observations. Lastly, an algorithm is designed which predicts the expression state from the generated state sequence. Performance is compared against several existing state-of-the-art FER systems on six publicly available datasets. A weighted average accuracy of 97% is achieved across all datasets.

Conditional Random Fields for Image Labeling

With the rapid development and application of CRFs (Conditional Random Fields) in computer vision, many researchers have made some outstanding progress in this domain because CRFs solve the classical version of the label bias problem with respect to MEMMs (maximum entropy Markov models) and HMMs (hidden Markov models). This paper reviews the research development and status of object recognition with CRFs and especially introduces two main discrete optimization methods for image labeling with CRFs: graph cut and mean field approximation. This paper describes graph cut briefly while it introduces mean field approximation more detailedly which has a substantial speed of inference and is researched popularly in recent years.