Maximum Entropy Classifier Research Articles

Permafrost Probability Mapping at a 30 m Resolution in Arxan Based on Multiple Characteristic Variables and Maximum Entropy Classifier

High-resolution permafrost mapping is an important direction in permafrost research. Arxan is a typical area with permafrost degradation and is situated on the southern boundary of the permafrost region in Northeast China. With the help of Google Earth Engine (GEE), the maximum entropy classifier (MaxEnt) is used for permafrost mapping using the land surface temperature (LST) of different seasons, deviation from mean elevation (DEV), solar radiation (SR), normalized difference vegetation index (NDVI), and normalized difference water index (NDWI) as the characteristic variables. The prior data of permafrost distribution were primarily based on 201 borehole data and field investigation data. A permafrost probability (PP) distribution map with a resolution of 30 m was obtained. The receiver operating characteristic (ROC) curve was used to test the distribution results, with an area under the curve (AUC) value of 0.986. The results characterize the distribution of permafrost at a high resolution. Permafrost is mainly distributed in the Greater Khingan Mountains (GKM) in the research area, which run from the northeast to the southwest, followed by low-altitude area in the northwest. According to topographic distribution, permafrost is primarily found on slope surfaces, with minor amounts present in peaks, ridges, and valleys. The employed PP distribution mapping method offers a suggestion for high-resolution permafrost mapping in permafrost degradation areas.

Machine Learning for Data Linkage

Data linkage traditionally uses deterministic and probabilistic methods. Alternatively, machine learning methods can be applied as classification algorithms, using the data to inform decisions. This project compared the quality, in terms of precision and recall, of traditional methods with selected machine learning methods when applied to a standard linkage problem.
 Two supervised methods, gradient boosted trees (GBT) and multiple layered perceptron classifier (MLPC), and one unsupervised method, maximum entropy classification (MEC), were implemented. The England and Wales 2021 Census to Census Coverage Survey (CCS) linkage was used as a gold-standard (GS) linked dataset to provide training samples for the supervised methods as well as testing samples for all methods. The F1 score (harmonic mean of precision and recall) was used to compare the performance of the models and to determine the optimal parameters and thresholds.
 The Splink implementation of Fellegi-Sunter with Expectation Maximisation was used as a baseline for comparison.
 The methods, trained on a sample of the GS, were used to link census and CCS data. All methods performed well with MEC achieving the highest precision (99.79%) but lowest recall (96.36%). The MLPC model achieved the highest F1 score (98.94%).
 To understand the implications of not retraining supervised models for each dataset, the models were also used to link Census to a health dataset. The supervised models were not retrained using the health data; instead, the optimised GS models were applied. MEC had the lowest precision (96.51%) but the highest recall (98.48%) and highest F1 score (97.49%). With F1 scores of 96.99% and 96.14% respectively, the GBT and MLPC supervised models were not far behind in performance, despite not being trained using health data.
 We have shown that machine learning methods can be used effectively for data linkage problems. Unsurprisingly, supervised models perform best when trained on and applied to the same data. Further research into generic training may allow us to use both supervised and unsupervised machine learning models for future data linkage.

A Word Sense Disambiguatin Approach For Romanian Language

The automatic word sense disambiguation has been a topic of interest since the 1950s. Meaning disambiguation is not an end in itself, it is an intermediate process, necessary at a certain level to use in natural language processing. It is obviously useful for applications that require language interpretation (message communication, human-machine interaction), but it is also used in fields whose main purpose is not to understand natural language. In this paper we propose an approach for word sense disambiguation for Romanian language, based on a supervised learning, a large number of features, and information gain to reduce the dimensions. The evaluation for the Romanian words subtask is produced by the extensive feature set in conjunction with a Maximum Entropy classifier and outcomes that are better than other similar researches.

An Efficient Internet Traffic Classification System Using Deep Learning for IoT

Internet of Things (IoT) defines a network of devices connected to the internet and sharing a massive amount of data between each other and a central location. These IoT devices are connected to a network therefore prone to attacks. Various management tasks and network operations such as security, intrusion detection, Quality-of-Service provisioning, performance monitoring, resource provisioning, and traffic engineering require traffic classification. Due to the ineffectiveness of traditional classification schemes, such as port-based and payload-based methods, researchers proposed machine learning-based traffic classification systems based on shallow neural networks. Furthermore, machine learning-based models incline to misclassify internet traffic due to improper feature selection. In this research, an efficient multilayer deep learning based classification system is presented to overcome these challenges that can classify internet traffic. To examine the performance of the proposed technique, Moore-dataset is used for training the classifier. The proposed scheme takes the pre-processed data and extracts the flow features using a deep neural network (DNN). In particular, the maximum entropy classifier is used to classify the internet traffic. The experimental results show that the proposed hybrid deep learning algorithm is effective and achieved high accuracy for internet traffic classification, i.e., 99.23%. Furthermore, the proposed algorithm achieved the highest accuracy compared to the support vector machine (SVM) based classification technique and k-nearest neighbours (KNNs) based classification technique.

A maximum entropy classification scheme for phishing detection using parsimonious features

Over the years, electronic mail (e-mail) has been the target of several malicious attacks. Phishing is one of the most recognizable forms of manipulation aimed at e-mail users and usually, employs social engineering to trick innocent users into supplying sensitive information into an imposter website. Attacks from phishing emails can result in the exposure of confidential information, financial loss, data misuse, and others. This paper presents the implementation of a maximum entropy (ME) classification method for an efficient approach to the identification of phishing emails. Our result showed that maximum entropy with parsimonious feature space gives a better classification precision than both the Naïve Bayes and support vector machine (SVM).

Comparative study between KNN and maximum entropy classification in sentiment analysis of menstrual cup

The rise of reuse, reduce, recycle movement impacts both in pollutants management and waste. Including menstrual waste by using the menstrual cup. However, the use of menstrual cups in Indonesia is not yet popular. Sentiment analysis is needed to see how the public thinks about the menstrual cup. Using 1108 data from Twitter, which was then labeled into positive and negative manually, the sentiment analysis stage was carried out using Maximum Entropy and K-Nearest Neighbor. These two were chosen because maximum entropy works by obtaining the best probability distribution, which closest to reality, and k-nearest neighbors work by classifying new objects based on attribute examples and training data. The implementation program uses PyCharm IDE and python programming language. From the research results, maximum entropy and k-nearest neighbors accuracy each are 84.6% and 83.7% with 1108 tweets. In the next research, a lexicon dictionary can be used to replace the manual labeling process.

Brain Tumour Detection by Gamma DeNoised Wavelet Segmented Entropy Classifier

Magnetic resonance imaging (MRI) is an essential tool for detecting brain tumours. However, identification of brain tumours in the early stages is a very complex task since MRI images are susceptible to noise and other environmental obstructions. In order to overcome these problems, a Gamma MAP denoised Strömberg wavelet segmentation based on a maximum entropy classifier (GMDSWS-MEC) model is developed for efficient tumour detection with high accuracy and low time consumption. The GMDSWS-MEC model performs three steps, namely pre-processing, segmentation, and classification. Within the GMDSWS-MEC model, the Gamma MAP filter performs the pre-processing task and achieves a significant increase in the peak signal-to-noise ratio by removing noisy artefacts from the input brain image. After pre-processing, Strömberg wavelet transform segmentation is carried out to partition the pre-processed image into a number of blocks based on the features extracted from the image. Finally, the maximum entropy classifier identifies and locates the tumour from the input image based on extracted features with high accuracy and minimal error rate. Using a number of MRI images, experimental evaluation and comparison of the proposed model and existing methods is carried out on the basis of four metrics: peak signal-to-noise ratio, tumour detection accuracy, error rate, and tumour detection time with respect to MRI image size. The proposed model offers superior performance in terms of all four metrics.

Model and Simulation of Maximum Entropy Phrase Reordering of English Text in Language Learning Machine

This paper proposes a feature extraction algorithm based on the maximum entropy phrase reordering model in statistical machine translation in language learning machines. The algorithm can extract more accurate phrase reordering information, especially the feature information of reversed phrases, which solves the problem of imbalance of feature data during maximum entropy training in the original algorithm, and improves the accuracy of phrase reordering in translation. In the experiment, they were combined with linguistic features such as parts of speech, words, and syntactic features extracted by using the syntax analyzer, and the maximum entropy classifier was used to predict translation errors, and the experimental verification was performed on the Chinese-English translation data set and compared. The experimental results show that different word posterior probabilities have a significant impact on the classification error rate, and the combination of linguistic features based on the word posterior probability can significantly reduce the classification error rate and improve the translation error prediction performance.

Burn severity analysis in Mediterranean forests using maximum entropy model trained with EO-1 Hyperion and LiDAR data

All ecosystems and in particular ecosystems in Mediterranean climates are affected by fires. Knowledge of the drivers that most influence burn severity patterns as well an accurate map of post-fire effects are key tools for forest managers in order to plan an adequate post-fire response. Remote sensing data are becoming an indispensable instrument to reach both objectives. This work explores the relative influence of pre-fire vegetation structure and topography on burn severity compared to the impact of post-fire damage level, and evaluates the utility of the Maximum Entropy (MaxEnt) classifier trained with post-fire EO-1 Hyperion data and pre-fire LiDAR to model three levels of burn severity at high accuracy. We analyzed a large fire in central-eastern Spain, which occurred on 16–19 June 2016 in a maquis shrubland and Pinus halepensis forested area. Post-fire hyperspectral Hyperion data were unmixed using Multiple Endmember Spectral Mixture Analysis (MESMA) and five fraction images were generated: char, green vegetation (GV), non-photosynthetic vegetation, soil (NPVS) and shade. Metrics associated with vegetation structure were calculated from pre-fire LiDAR. Post-fire MESMA char fraction image, pre-fire structural metrics and topographic variables acted as inputs to MaxEnt, which built a model and generated as output a suitability surface for each burn severity level. The percentage of contribution of the different biophysical variables to the MaxEnt model depended on the burn severity level (LiDAR-derived metrics had a greater contribution at the low burn severity level), but MaxEnt identified the char fraction image as the highest contributor to the model for all three burn severity levels. The present study demonstrates the validity of MaxEnt as one-class classifier to model burn severity accurately in Mediterranean countries, when trained with post-fire hyperspectral Hyperion data and pre-fire LiDAR.

A patch-based algorithm for global and daily burned area mapping

Increasing availability of dense time series of moderate spatial resolution satellite data for mapping global burned areas calls for mapping algorithms designed to easily integrate data at different spatial and temporal resolutions, irrespective of particular grid constraints. In this paper, we describe a novel hybrid approach for global burned area mapping that combines active fire data and time series of surface reflectance using graphs, which provide a flexible and efficient way of extracting spatiotemporal consistent patches.Our approach has three main steps. Firstly, we analyze burn-sensitive vegetation index time series to determine for each location a set of events, which are the dates for which the spectral-temporal signal indicates the possibility of a burn. Secondly, we explore the spatiotemporal distribution of all events and active fires to determine a subset of events with strong evidence of corresponding to burned areas. Those events are used as positive occurrences for training a one-class maximum entropy classifier and obtain, for each candidate event, a likelihood of it actually corresponding to a burn. Finally, we build a graph that combines all previous information, from which we extract spatiotemporal patches of densely connected events. Patches with strong evidence of burning determine the burned area map at any given time period.This research is part of the European Space Agency's Climate Change Initiative (ESA-CCI) and aims ultimately at generating Sentinel-3 daily global burned area products at 500 m spatial resolution. Towards that end, we test our approach with spatially and spectrally similar MODIS gridded surface reflectance (MOD/MYD09GA), as well as non-gridded active fire (MCD14ML) 2008 data and CCI global land cover maps. Using 105 independent Landsat fire reference perimeters to validate global results, we show that our algorithm applied to MOD/MYD09GA data (PT-M09) has very similar accuracy (44%) measured by the Dice coefficient compared with MCD64A1 v006 (45%). Moreover, PT-M09 exhibits a higher commission error but a lower omission error than MCD64A1. Due to their coarse resolution, this kind of product cannot capture very small burn areas. The bias relative to the reference burned area indicates that the algorithm presented in the current study underestimates burned area by 14% of the area actually burned according to reference data, which is lower than the underestimation in MCD64A1 (28%).We also analyzed the temporal accuracy of the patch based algorithm and concluded that the average time difference to active fires detection is 3.43 days, which is similar to MCD64A1 (3.75 days). Finally, we performed a sensitivity analysis which shows that total mapped burned areas varies only 3% when the main algorithm threshold that separates “burned” and “unburned” patches varies from quantile 45% to 55%.

Lessons Learned from an Extreme Heat Event using ACES for Situational Awareness, Ontario, Canada

ObjectiveTo describe the lessons learned for public health decision-makers from an analysis of Acute Care Enhanced Surveillance (ACES) data for the heatwaves experienced in Ontario, Canada in the summer of 2018.IntroductionThe Acute Care Enhanced Surveillance (ACES) system provides syndromic surveillance for Ontario’s acute care hospitals. ACES receives over 99% of acute care records for emergency department (ED) visits; mean daily volume is 17,500 visits. ACES uses a maximum entropy classifier and generates more than 80 standard syndromes, fifteen of which are actively monitored for aberrational activity and are considered of higher public health relevance, including RESP (respiratory infection, non-croup), ILI (influenza-like illness), TOX (toxicological, chemical/drug exposure), AST (asthma), OPI (opioid exposure), CELL (cellulitis), GASTRO (gastroenteritis), ENVIRO (environmental, heat/cold exposure), MH (mental health), EOH (alcohol intoxication), DERM (rash), and SEP (bacteremia, sepsis).Syndromic surveillance provides a salient source of public health surveillance during extreme heat events; monitoring real-time ED visits can inform local public health authorities of health impacts, provide situation awareness to initiate and/or inform public health response, and help decision-makers allocate resources according to geographic (or demographic) vulnerability. While the use of syndromic surveillance has been well-characterized to monitor infectious disease outbreaks, its use to monitor the heat-health impacts is relatively novel for ACES users, specifically local public health authorities. This report describes the the data collected during an extended extreme heat event in Ontario, Canada, to highlight the value of syndromic surveillance during extreme heat events and make recommendations regarding incorporating ACES data into routine workflows.MethodsTemperature data were retrieved from Environment Canada historical databases for mid-June to mid-July 2018. Aggregate counts per day for total ED visits and and for individual syndromes were retrieved from ACES databases. Descriptive statistics were used to analyze all datasets.ResultsAn extreme heat event occurred in the southern region of Ontario in early summer, 2018. Environment Canada issues heat warnings for regions throughout Canada according to region-specific criteria; for southern Ontario, heat warnings are issued when 2 or more consecutive days of daytime maximum temperatures are expected to reach 31°C or when 2 or more consecutive days of humidex values are expected to reach 40. Extended heat warnings are issued when the event lasts beyond 2 days. An extended heat event occurred June 29 to July 5, 2018. Although the region is large, temperature data from Environment Canada’s climate monitoring station at Toronto’s Pearson Airport are shown (Figure 1) as an example of the temperatures observed for this time period in the region.ConclusionsLessons learned from an analysis of ACES data during an extreme heat event:1. The ENVIRO syndrome provides real-time monitoring of the health impacts during a heat event and may provide proxy for estimating the indirect effects of heat (e.g., impacts on chronic conditions). Public health authorities can monitor local health impacts during an extreme heat event.2. Patients seeking help at the ED do not appear to be skewed in acuity, sex nor age. This does not necessarily reflect the population that experiences the greatest impact from extreme heat, but rather those that are seeking help at the ED for the direct effects of heat. That said, an increase in ENVIRO counts does not indicate whether the increase is due to greater exposure to the heat (or sun), engaging in vigorous outdoor activity during the event (recreational or occupational), or lack of access to air conditioning.3. ED visits for ENVIRO can be geolocated to determine areas experiencing greater health impacts. This may allow allocation of resources to specifically address vulnerabilities. ACES has built-in mapping capabilities that allows a geovisualization of the home addresses for patients. Furthermore, aggregate counts for relevant syndromes are available for registered users on the Public Health Information Management System (PHIMS), a web-accessible GIS tool for situational awareness that gives public health decision-makers access to real time health impacts in concert with demographics, weather, and other emergency management information.

Knowledge Generation Using Sentiment Classification Involving Machine Learning on E-Commerce

Sentiment analysis manages the computational treatment of conclusion, notion, and content subjectivity. In this article, three sentiment classes such as positive, negative and neutral emotions have been demonstrated by appropriate features from raw unstructured data followed by data preprocessing steps. Applying best in class social analytics methodology to examine the sentiments embedded with purchaser remarks, encourages both producer and individual customers. Machine learning methods such as Naïve Bayes, maximum entropy classification, Deep Neural Networks were used upon the data, extracted from some websites such as Samsung and Apple for sentiment classification. In the online business arena, the application of sentiment classification explores a great opportunity. The subsidy of such an investigation is that associations can apply the proposed social examination framework to exploit the entire social information on the web and therefore improve their proper blueprint promoting strategies corresponding business.

IWD Based Feature Selection Algorithm for Sentiment Analysis

Feature selection methods aim to improve the classification performance by eliminating non-valuable features. In this paper, our aim is to apply a recent optimization technique namely the Intelligent Water Drops (IWD) algorithm to select best features for sentiment analysis. We investigate the classification performances of our proposed IWD based feature selection method by comparing one of the well-known feature selection method using Maximum Entropy classifier. Experimental results show that Intelligent Water Drops based feature selection method outperforms than ReliefF method for sentiment analysis.DOI: 10.5755/j01.eie.25.1.22736

Word Sense Disambiguation based on Maximum Entropy Classifier

Word Sense Disambiguation based on Maximum Entropy Classifier

Sentiment analysis on Bahasa Indonesia tweets using Unibigram models and machine learning techniques

Sentiment analysis on English tweets has its challenges. In addition to frequent use of the informal language, the words used are usually less consistent, contain abbreviations, and mixed with local languages. In this study, we combined n-grams feature selection models, i.e., unigrams, bigrams, and unibigram (1 + 2-grams) to analyze the public opinion of the Bahasa Indonesia tweets about the presidential candidates of the Republic of Indonesia in the 2014 presidential election. The experiment was carried out using Naive Bayes classifiers, Maximum Entropy classifiers, and Support Vector Machines with and without stop words removal and stemming in pre-processed tweet documents. The experiment results show the best performance is achieved by Naïve Bayes classifiers with unibigram feature models without removing the stop words and stemming process. This method achieved high performance with the precision and recall up to 85.50% indicating that automatic sentiment analysis of tweet documents using well-known supervised learning methods is feasible for the Indonesian language. More interestingly is the fact that stop words removal and stemming process on the corpus made the classification performances worse compared with the corpus that experienced cleansing only.

Research on multimodal human-robot interaction based on speech and gesture

This paper presents a multimodal human-robot interaction based on fusion of speech and gesture. In the interface, a robot control command system is designed, which can transform the speech and gesture of users into commands that the robot can execute. Microsoft speech SDK is used in this system to collect the speech of the operator. Then, a corpus-based algorithm of maximum entropy classification for natural language understanding is employed to generate commands. Leap Motion is employed to capture the gesture of operator in this system. Interval Kalman Filter (IKF) is used to estimate the measured data to reduce the inherent noise of the sensor. The advantage of the proposed method is that the combination of speech and gesture makes the human-robot interaction more convenient and direct. Finally, a series of experiments were carried out to validate our method, and proved that it performed better than the other proposed methods.

A Machine Learning Aided Systematic Review and Meta-Analysis of the Relative Risk of Atrial Fibrillation in Patients With Diabetes Mellitus.

Background: Meta-analysis is a widely used tool in which weighted information from multiple similar studies is aggregated to increase statistical power. However, the exponential growth of publications in key areas of medical science has rendered manual identification of relevant studies increasingly time-consuming. The aim of this work was to develop a machine learning technique capable of robust automatic study selection for meta-analysis. We have validated this approach with an up-to-date meta-analysis to investigate the association between diabetes mellitus (DM) and new-onset atrial fibrillation (AF).Methods: The PubMed online database was searched from 1960 to September 2017 where 4,177 publications that mentioned both DM and AF were identified. Relevant studies were selected as follows. First, publications were clustered based on common text features using an unsupervised K-means algorithm. Clusters that best matched the selected set of potentially relevant studies (a “training” set of 139 articles) were then identified by using maximum entropy classification. The 139 articles selected automatically on this basis were screened manually to identify potentially relevant studies. To determine the validity of the automated process, a parallel set of studies was also assembled by manually screening all initially searched publications. Finally, detailed manual selection was performed on the full texts of the studies in both sets using standard criteria. Quality assessment, meta-regression random-effects models, sensitivity analysis and publication bias assessment were then conducted.Results: Machine learning-assisted screening identified the same 29 studies for meta-analysis as those identified by using manual screening alone. Machine learning enabled more robust and efficient study selection, reducing the number of studies needed for manual screening from 4,177 to 556 articles. A pooled analysis using the most conservative estimates indicated that patients with DM had ~49% greater risk of developing AF compared with individuals without DM. After adjusting for three additional risk factors i.e., hypertension, obesity and heart disease, the relative risk was 23%. Using multivariate adjusted models, the risk for developing AF in patients with DM was similar for all DM subtypes. Women with DM were 24% more likely to develop AF than men with DM. The risk for new-onset AF in patients with DM has also increased over the years.Conclusions: We have developed a novel machine learning method to identify publications suitable for inclusion in meta-analysis.This approach has the capacity to provide for a more efficient and more objective study selection process for future such studies. We have used it to demonstrate that DM is a strong, independent risk factor for AF, particularly for women.

Detailing Sentiment Analysis to Consider Entity Aspects: An Approach for Portuguese Short Texts

Sentiment analysis is useful for identifying trends, or for discovering user preferences, which can later be applied to campaign targeting or recommendations. In this paper, we describe an approach to classify the sentiment polarity regarding aspects, and how this technique was used in a previous system, for short texts in Portuguese, giving it greater sensitivity to detail. Aspect extraction is done by locating candidates for aspect as expressions having a relationship with the entity and possibly some polarized term, through rules based on POS tags. For each aspect, the sentiment polarity is determined by a Maximum Entropy classifier, whose features depend on the entity mention, on the aspect and its support text, including negation detection, bigrams, POS tags, and sentiment lexicon-based polarity clues. For aspect sentiment, our classifier evaluation indicated a precision of 68% for the positive class and 73% for the negative class, with the dataset used in our research.

Sentiment Extraction from Naturalistic Video

Sentiment analysis on video is quite an unexplored field of research wherein the emotion and sentiment of the speaker are extracted by processing the frames, audio and text obtained from the video. In recent times, sentiment analysis from naturalistic audio has been an upcoming field of research. This is typically done by performing automatic speech recognition on audio, followed by extracting the sentiment exhibited by the speaker. On the other hand, techniques for extracting sentiments from text are quite developed and tech giants have already optimized these methods to process large amounts of customer review, feedback and reactions. In this paper, a new model for sentiment analysis from audio is proposed which is a hybrid of Keyword Spotting System (KWS) and Maximum Entropy (ME) Classifier System. This model is developed with the aim to outperform other conventional classifiers and to provide a single integrated system for audio and text processing. In addition, a web application for dynamic processing of YouTube videos is described. The WebApp provides an index-based result for each phrase that is detected in the video. Often, the emotion of the viewer of a video corresponds to its content. In this regard, it is useful to map these emotions to the text transcript of the video and assign a suitable weight to it while predicting the sentiment that the speaker exhibits. This paper describes such an application that was developed to analyze facial expressions using Affdex API. Thus, using the combined statistics from all the three aforementioned components, a robust and portable system for emotion detection is obtained that provides accurate predictions and can be deployed on any modern systems with minimal configuration changes.

An improved algorithm for sentiment analysis based on maximum entropy

Sentiment analysis is an important field of study in natural language processing. In the massive data and irregular data, sentiment classification with high accuracy is a major challenge in sentiment analysis. To address this problem, a novel maximum entropy-PLSA model is proposed. In this model, we first use the probabilistic latent semantic analysis to extract the seed emotion words from the Wikipedia and the training corpus. Then features are extracted from these seed emotion words, which are the input of the maximum entropy model for training the maximum entropy model. The test set is processed similarly into the maximum entropy model for emotional classification. Meanwhile, the training set and the test set are divided by the K-fold method. The maximum entropy classification based on probabilistic latent semantic analysis uses important emotional classification features to classify words, such as the relevance of words and parts of speech in the context, the relevance with degree adverbs, the similarity with the benchmark emotional words and so on. The experiments prove that the classification method proposed by this paper outperforms the compared methods.