Standard Performance Metrics Research Articles

A Long‐term Analysis of Physician Triage Screening in the Emergency Department

The problem of emergency department (ED) crowding is well recognized; however, little data exist on the sustainability of potential solutions, including physician triage and screening. The authors hypothesized that a physician triage screening program (Supplemented Triage and Rapid Treatment [START]) sustainably improves standard ED performance metrics. This retrospective, observational, before-and-after study compared performance measures over 4 years in a tertiary care urban academic medical center with approximately 90,000 annual ED visits. Patients seen between December 2006 and November 2010 were included. Outcome measures included length of stay (LOS) for ED patients, percentage of patients who left without completing assessment (LWCA), percentage of patients treated and dispositioned by START without using monitored beds, and door-to-room time. Descriptive statistics were used. Median LOS for START patients was 56 minutes/patient lower when comparing 2010 to 2007 (p < 0.0001) and for non-START patients 22 minutes/patient lower (p < 0.0001). The percentage of patients who LWCA decreased from 4.8% to 2.9% (p < 0.0001) during the same time period. In START's first half-year, 18% of patients were discharged without using monitored beds. This increased to 29% by year 3. In addition, median door-to-room time decreased from 18.4 to 9.9 minutes during the same 3-year interval. Physician screening appears to provide sustainable improvements in ED performance metrics including ED LOS, percentage of patients who LWCA, door-to-room time, and percentage of patients treated without using a monitored bed, despite increasing ED volume. Physician screening delivers additional incremental benefits for several years after implementation and can effectively increase ED capacity by allowing emergency physicians to more efficiently use monitored beds.

The Value of Hiring through Referrals

Using unique personnel data from nine large firms in three industries, we document five consistent facts about hiring through employee referral networks. First, referred applicants have similar skill characteristics to non-referred applicants, both observable-to-the-firm (e.g., schooling) and unobservable-to-the-firm (e.g., cognitive and non-cognitive ability), but are more likely to be hired, more likely to accept job offers, and have higher pre-job assessment scores. Second, referred workers have similar skill characteristics to non-referred workers. Third, referred workers are less likely to quit and are more productive, but only on rare high-impact performance metrics; on most standard non-rare performance metrics, referred and non-referred workers perform similarly. Fourth, referred workers have slightly higher wages, but yield substantially higher profits per worker. Fifth, workers who make referrals have higher productivity than others, are less likely to quit after making a referral, and refer those like themselves on particular productivity metrics. Differences between referred and non-referred workers tend to be larger at low-tenure levels; for young, Black, and Hispanic workers; and in strong labor markets. No leading class of theories can alone account for all or most of these results, leading us to suggest several theoretical extensions.

Named entity recognition of follow-up and time information in 20 000 radiology reports

To develop a system to extract follow-up information from radiology reports. The method may be used as a component in a system which automatically generates follow-up information in a timely fashion. A novel method of combining an LSP (labeled sequential pattern) classifier with a CRF (conditional random field) recognizer was devised. The LSP classifier filters out irrelevant sentences, while the CRF recognizer extracts follow-up and time phrases from candidate sentences presented by the LSP classifier. The standard performance metrics of precision (P), recall (R), and F measure (F) in the exact and inexact matching settings were used for evaluation. Four experiments conducted using 20,000 radiology reports showed that the CRF recognizer achieved high performance without time-consuming feature engineering and that the LSP classifier further improved the performance of the CRF recognizer. The performance of the current system is P=0.90, R=0.86, F=0.88 in the exact matching setting and P=0.98, R=0.93, F=0.95 in the inexact matching setting. The experiments demonstrate that the system performs far better than a baseline rule-based system and is worth considering for deployment trials in an alert generation system. The LSP classifier successfully compensated for the inherent weakness of CRF, that is, its inability to use global information.

Assessment of performance of survival prediction models for cancer prognosis

BackgroundCancer survival studies are commonly analyzed using survival-time prediction models for cancer prognosis. A number of different performance metrics are used to ascertain the concordance between the predicted risk score of each patient and the actual survival time, but these metrics can sometimes conflict. Alternatively, patients are sometimes divided into two classes according to a survival-time threshold, and binary classifiers are applied to predict each patient’s class. Although this approach has several drawbacks, it does provide natural performance metrics such as positive and negative predictive values to enable unambiguous assessments.MethodsWe compare the survival-time prediction and survival-time threshold approaches to analyzing cancer survival studies. We review and compare common performance metrics for the two approaches. We present new randomization tests and cross-validation methods to enable unambiguous statistical inferences for several performance metrics used with the survival-time prediction approach. We consider five survival prediction models consisting of one clinical model, two gene expression models, and two models from combinations of clinical and gene expression models.ResultsA public breast cancer dataset was used to compare several performance metrics using five prediction models. 1) For some prediction models, the hazard ratio from fitting a Cox proportional hazards model was significant, but the two-group comparison was insignificant, and vice versa. 2) The randomization test and cross-validation were generally consistent with the p-values obtained from the standard performance metrics. 3) Binary classifiers highly depended on how the risk groups were defined; a slight change of the survival threshold for assignment of classes led to very different prediction results.Conclusions1) Different performance metrics for evaluation of a survival prediction model may give different conclusions in its discriminatory ability. 2) Evaluation using a high-risk versus low-risk group comparison depends on the selected risk-score threshold; a plot of p-values from all possible thresholds can show the sensitivity of the threshold selection. 3) A randomization test of the significance of Somers’ rank correlation can be used for further evaluation of performance of a prediction model. 4) The cross-validated power of survival prediction models decreases as the training and test sets become less balanced.

Economic design of $\bar X$ &amp; S control charts based on Taguchi’s loss function and its optimization

Much research effort has been devoted to economic design of $\bar X$ & S control charts, however, there are some problems in usual methods. On the one hand, it is difficult to estimate the relationship between costs and other model parameters, so the economic design method is often not effective in producing charts that can quickly detect small shifts before substantial losses occur; on the other hand, in many cases, only one type of process shift or only one pair of process shifts are taken into consideration, which may not correctly reflect the actual process conditions. To improve the behavior of economic design of control chart, a cost & loss model with Taguchi’s loss function for the economic design of $\bar X$ & S control charts is embellished, which is regarded as an optimization problem with multiple statistical constraints. The optimization design is also carried out based on a number of combinations of process shifts collected from the field operation of the conventional control charts, thus more hidden information about the shift combinations is mined and employed to the optimization design of control charts. At the same time, an improved particle swarm optimization (IPSO) is developed to solve such an optimization problem in design of $\bar X$ & S control charts, IPSO is first tested for several benchmark problems from the literature and evaluated with standard performance metrics. Experimental results show that the proposed algorithm has significant advantages on obtaining the optimal design parameters of the charts. The proposed method can substantially reduce the total cost (or loss) of the control charts, and it will be a promising tool for economic design of control charts.

Human-Oriented Control for Haptic Teleoperation

Haptic teleoperation enables the human to perform manipulation tasks in distant, scaled, hazardous, or inaccessible environments. The human closes the control loop sending haptic command signals to and receiving haptic feedback signals from the remote teleoperator. The main research question is how to design the control such that human decision making and action is supported in the best possible way while ensuring robust operation of the system. The human in the loop induces two major challenges for control design: 1) the dynamics of the human operator and the teleoperation system are tightly coupled, i.e., stability of the overall system is affected by the human operator dynamics; and 2) the performance of the teleoperation system is subjectively evaluated by the human, which typically means that standard control performance metrics are not suitable. This paper discusses recent control design successes in the area of haptic teleoperation. In particular, the importance and need of dynamic human haptic closed-loop behavior models and human perception models for the further improvement of haptic teleoperation systems is highlighted and discussed for real-world problem domains.

Application of Data Mining Techniques in Weather Prediction and Climate Change Studies

Weather forecasting is a vital application in meteorology and has been one of the most scientifically and technologically challenging problems around the world in the last century. In this paper, we investigate the use of data mining techniques in forecasting maximum temperature, rainfall, evaporation and wind speed. This was carried out using Artificial Neural Network and Decision Tree algorithms and meteorological data collected between 2000 and 2009 from the city of Ibadan, Nigeria. A data model for the meteorological data was developed and this was used to train the classifier algorithms. The performances of these algorithms were compared using standard performance metrics, and the algorithm which gave the best results used to generate classification rules for the mean weather variables. A predictive Neural Network model was also developed for the weather prediction program and the results compared with actual weather data for the predicted periods. The results show that given enough case data, Data Mining techniques can be used for weather forecasting and climate change studies.

Collecting Distributed Performance Data with Dataheap: Generating and Exploiting a Holistic System View

The task of performance analysis and optimization grows more and more challenging with the increasing scale and complexity of large computing systems. The need for a holistic system analysis becomes apparent when traditional approaches do not collect the information that is required to investigate performance penalties caused by shared system resources. We have developed a distributed approach that is able to collect and process performance data from shared system resources. We call our software implementation of this approach Dataheap and have integrated it with a traditional program tracing facility. In this paper we describe the needs that have driven this development as well as connections to related projects. Dataheap is based on a threaded server, distributed agents that collect performance data, a storage backend that makes use of different databases, and access libraries that allow external systems to retrieve current and historic performance data. The server subsequently processes incoming performance data and allows to create secondary metrics on the fly which helps to transform individual system characteristics to standard performance metrics. Finally, we briefly illustrate how this approach has enhanced our performance debugging capabilities as well as our research on energy effcient computing.

Discrete hybrid PSO algorithm for design of laminate composites with multiple objectives

This article presents a multi-objective discrete hybrid adaptive swarm intelligence algorithm for combinatorial optimization and applied for design optimization of fiber-reinforced composite structures. An approach is presented in this article to integrate a Pareto dominance concept into the adaptive PSO algorithm developed earlier by the first author in order to handle multi-objective optimization problems. In fact, a hybrid version of the adaptive PSO algorithm is now proposed in this article for multi-objective optimization by integrating with discrete as well as continuous neighborhood search algorithms. Further, an external archive technique is also integrated in order to collect the historical Pareto optimal solutions. The design constraints are handled in this article by treating them as additional objectives. Numerical studies have been carried out through the optimization of a hybrid fiber-reinforced composite plate, laminate composite cylindrical shell, and also a pressure vessel problem with varied number of design objectives. Standard performance metrics are employed to evaluate the performance of the proposed multi-objective hybrid PSO algorithm with both discrete as well as continuous neighborhood search algorithms. Studies clearly favor hybrid PSO with variable-depth neighborhood search algorithm. Finally, comparisons have also been made with other popular evolutionary algorithms like NSGA-II, PAES, Micro-GA, and SPEA2 to demonstrate the effectiveness of the proposed hybrid multi-objective PSO algorithm.

Economic and statistical design of and S control charts using an improved multi-objective particle swarm optimisation algorithm

This research work embellishes a cost and loss model with Taguchi's loss function to incorporate the avoidable surplus quality losses due to failure to detect the out-of-control state for the economic and statistical optimisation design of and S control charts. The optimisation design considered as a multi-objective decision-making problem is carried out based on the adequate knowledge related to the process shifts, which is extracted from the field operation of conventional control charts. An improved crowding distance based fuzzy multi-objective particle swarm optimisation (CD-FMOPSO) is developed to solve such a multi-objective problem in optimisation design of and S control charts. The proposed CD-FMOPSO algorithm is first tested for several benchmark problems taken from the literature and evaluated with standard performance metrics. The test results show that the proposed algorithm does not have any difficulty in achieving well-spread Pareto optimal solutions with good convergence to true Pareto optimal front for multi-objective optimisation problems. Finally, to facilitate ease in decision making for quality control managers, a simple but effective decision making approach was introduced. The result obtained indicates that the proposed approach may be a promising tool for economic and statistical optimisation design of control charts.

An Abstract Semantically Rich Compiler Collocative and Interpretative Model for OpenMP Programs

To understand the behavior of OpenMP programs, special tools and adaptive techniques are needed for performance analysis. However, these tools provide low-level profile information at the assembly and functions boundaries via instrumentation at the binary or code level, which are very hard to interpret. Moreover, to compare different OpenMP-enabled compilers, there is no systematic methodology that provides an easy comparison. Hence, in this paper, we propose a new model for OpenMP-enabled compilers that assesses the performance differences in well-defined formulations by dividing OpenMP program conditions into four distinct states which account for all the possible cases that an OpenMP program can take. The model works as a first-level inspector to reason about the effect of compiler performance on every state in an unobtrusive and informative way. In addition, an improved version of the standard performance metrics is proposed: speedup, overhead and efficiency based on the model categorization that is state's aware. The evaluation shows that the improved version is more accurate and insightful in terms of OpenMP implementation. Moreover, an algorithmic approach to find patterns between OpenMP compilers is proposed, which is verified along with the model formulations experimentally. We also show the mathematical model behind the optimum performance for any OpenMP program.

Multi-objective optimization of multi-pass face milling using particle swarm intelligence

In this paper, to facilitate manufacturing engineers have more control on the machining operations, the optimization issue of machining parameters is handled as a multi-objective optimization problem. The optimization strategy is to simultaneously minimize production time and cost and maximize profit rate meanwhile subject to satisfying the constraints on the machine power, cutting force, machining speed, feed rate, and surface roughness. An efficient fuzzy global and personal best-mechanism-based multi-objective particle swarm optimization (F-MOPSO) to optimize the machining parameters is developed to solve such a multi-objective optimization problem in optimization of multi-pass face milling. The proposed F-MOPSO algorithm is first tested on several benchmark problems taken from the literature and evaluated with standard performance metrics. It is found that the F-MOPSO does not have any difficulty in achieving well-spread Pareto optimal solutions with good convergence to true Pareto optimal front for multi-objective optimization problems. Upon achieving good results for test cases, the algorithm was employed to a case study of multi-pass face milling. Significant improvement is indeed obtained in comparison to the results reported in the literatures. The proposed scheme may be effectively employed to the optimization of machining parameters for multi-pass face milling operations to obtain efficient solutions.

On Demand Temporary Parallel Route Recovery for Frequent Link Failure in Adhoc Networks

Adhoc network is collection of wireless mobile nodes dynamically forming a temporary network without the use of existing network infrastructure of centralized administration. Each node acts as a router by itself and forwards all the packets which it receives. This concept is basically useful when there is a very high link failure due to migration of nodes from the available network region. There is a frequent link failure in adhoc networks, which causes packet to be lost or packets doubts to reach destination. In this scenario a different mechanism and scheme is proposed and implemented to make the important time critical data like real time or voice data to reach the destination without any loss. The mechanism used is a special propagation which propagates a unique kind of parallel route discovery for real time application scenario to send the time critical data safely. The scheme used is temporary parallel route recovery builds a temporary parallel path between the nodes during link failure. The important node then forwards the buffered packets to the destination without any loss which is on-demand based on type of information a node forwards. Special buffer is allocated and simulated by using a simulation tool NS-2. This extensive mode is compared with standard models performance metrics such as packet delivery ration, routing overheads and average delay are analyzed.

Perceptual learning of oriented gratings as revealed by classification images

Classification image analysis is a psychophysical technique in which noise components of stimuli are analyzed to produce an image that reveals critical features of a task. Here we use classification images to gain greater understanding of perceptual learning. To achieve reasonable classification images within a single session, we developed an efficient classification image procedure that employed designer noise and a low-dimensional stimulus space. Subjects were trained across ten sessions to detect the orientation of a grating masked in noise, with an eleventh, test, session conducted using a stimulus orthogonal to the trained stimulus. As with standard perceptual learning studies, subjects showed improvements in performance metrics of accuracy, threshold, and reaction times. The clarity of the classification images and their correlation to an ideal target also improved across training sessions in an orientation-specific manner. Furthermore, image-based analyses revealed aspects of performance that could not be observed with standard performance metrics. Subjects with threshold improvements learned to use pixels across a wider area of the image, and, apposed to subjects without threshold improvements, showed improvements in both the bright and dark parts of the image. We conclude that classification image analysis is an important complement to traditional metrics of perceptual learning.

Sci-Sat AM(1): Planning - 08: Estimating Planning Target Volume Margins for Fractionated Stereotactic Radiotherapy on Perfexion

The purpose of this study was to estimate planning target volume (PTV) margins for frame‐based Perfexion (PFX) SRT using the eXtend™ system's relocatable head frame (RHF). Patients with large brain metastases are currently undergoing hypofractionated (3 fractions) SRT on PFX enrolled on a phase 1 dose‐escalation clinical trial. In prior investigation, the performance of the RHF was quantified using cone‐beam CT (CBCT) in fourteen patients undergoing linac‐based SRT (median: 30 treatment fractions). Standard performance metrics — group mean (μ), systematic (Σ) and random (σ) uncertainties — were determined for frame‐guided positioning and intra‐fraction motion. A published margin‐determination formula (2.5*Σ) +0.7*σ) was used to estimate the PTV margin. An additional factor of (σ/√3) was added to the systematic component of the formula when initially designing the PTV for 3 fractions in PFX‐SRT. To more accurately account for PFX dose distributions and only 3 treatment fractions, a population‐based stochastic modeling approach is being developed to refine the PTV margin for hypofractionated PFX‐SRT. For frame‐guided SRT (30 fractions), the post‐correction positioning performance estimates were μ(position) = {0.1,−0.2,−0.6}mm, Σ(position) = {0.2;0.8;0.6}mm, and σ(position) = {0.3;0.6;0.4}mm in {Right; Superior;Anterior}. For intra‐fraction motion, μ(motion) = {−0.1;−0.1;0.0}mm, Σ(motion) = {0.2;0.2;0.1}mm, and σ(motion)={0.2;0.4;0.2}mm. The margin formula indicated an expansion of {1.0;2.6;1.8}mm and {1.6;3.1;2.3}mm for 30 fractions and 3 fractions, respectively. For three patients treated to date on PFX, μ(position) = (0.2mm;−0.9mm;−0.8mm). To ensure that the GTV receives the prescription dose, PTV margins have been calculated to account for the geometric uncertainties present in PFX‐SRT. The margins will be reviewed as more data are collected, RHF refinements are made, and stochastic‐based modeling is used.

A Meta‐Heuristic Algorithm for Multi‐Objective Optimal Design of Hybrid Laminate Composite Structures

Abstract: In this article, we propose a meta‐heuristic algorithm for solving multi‐objective combinatorial optimization problems. The proposed multi‐objective combinatorial optimization algorithm is developed by combining the good features of popular guided local search algorithms like simulated annealing (SA) and tabu search (TS). It has been organized as a multiple start algorithm to maintain a good balance between intensification and diversification. The proposed meta‐heuristic algorithm is evaluated by solving the stacking sequence optimization of hybrid fiber‐reinforced composite plate, cylindrical shell, and pressure vessel problems. The standard performance metrics for evaluating multi‐objective optimization algorithms are used to demonstrate the effectiveness of the proposed algorithm over other popular evolutionary algorithms like Nondominated Sorting Genetic Algorithms (NSGA‐II), Pareto Archived Evolutionary Strategy (PAES), micro‐GA, and Multi‐Objective Particle Swarm Optimization (MOPSO).

Bedside Ultrasonography to Identify Hip Effusions in Pediatric Patients

We determine whether pediatric emergency physicians can use bedside ultrasonography to accurately identify hip effusions in pediatric patients. This was a prospective study conducted in the emergency department (ED) of an urban tertiary care freestanding pediatric hospital. A convenience sample of children younger than 18 years and who required hip ultrasonography as part of their ED evaluation was enrolled. Pediatric emergency physicians with focused ultrasonographic training performed bedside ultrasonography on patients' symptomatic and contralateral hips and categorized the findings as "effusion" or "no effusion," according to a priori definitions. Physicians rated their confidence for each bedside ultrasonographic result on a scale of 1 (not confident) to 5 (very confident). Bedside ultrasonographic results were compared with the radiology department's ultrasonographic results, which were considered the criterion standard. Standard performance metrics (sensitivity, specificity, and positive and negative predictive values) were calculated. Three physicians enrolled patients. Twenty-eight patients were enrolled, and 55 hips were studied. In all hips (both symptomatic and contralateral), bedside ultrasonography had a sensitivity of 80% (95% confidence interval [CI] 51% to 95%), a specificity of 98% (95% CI 85% to 99%), a positive predictive value of 92% (95% CI 62% to 99%), and a negative predictive value of 93% (95% CI 79% to 98%). In the 28 symptomatic hips, bedside ultrasonography had a sensitivity of 85% (95% CI 54% to 97%), a specificity of 93% (95% CI 66% to 99%), a positive predictive value of 92% (95% CI 60% to 99%), and negative predictive value of 88% (95% CI 60% to 98%). When physician self-rated confidence was high, the sensitivity of bedside ultrasonography in symptomatic hips was 90% (95% CI 54% to 99%), the specificity was 100% (95% CI 70% to 100%), the positive predictive value was 100% (95% CI 63% to 100%), and the negative predictive value was 92% (95% CI 62% to 99%). With focused training, pediatric emergency physicians were able to use bedside ultrasonography to identify hip effusions in pediatric ED patients.

Subchronic feeding study of grain from herbicide-tolerant maize DP-Ø9814Ø-6 in Sprague-Dawley rats

This 13-week feeding study conducted in Sprague-Dawley rats evaluated the potential health effects from long-term consumption of a rodent diet formulated with grain from genetically modified (GM), herbicide-tolerant maize DP-Ø9814Ø-6 (98140; trade name Optimum® GAT® (Optimum® GAT® is a registered trademark of Pioneer Hi-Bred)). Metabolic inactivation of the herbicidal active ingredient glyphosate was conferred by genomic integration and expression of a gene-shuffled acetylase coding sequence, gat4621, from Bacillus licheniformis; tolerance to acetolactate synthase (ALS) inhibiting herbicides was conferred by overexpression of a modified allele (zm-hra) of the endogenous maize ALS enzyme that is resilient to inactivation. Milled maize grain from untreated (98140) and herbicide-treated (98140+Gly/SU) plants, the conventional non-transgenic, near-isogenic control (091), and three commercial non-transgenic reference hybrids (33J56, 33P66, and 33R77) was substituted at concentrations of 35–38% w/w into a common rodent chow formula (PMI® Nutrition International, LLC Certified Rodent LabDiet® 5002) and fed to rats (12/sex/group) for at least 91 consecutive days. Compared with rats fed diets containing grain from the conventional near-isogenic control maize, no adverse effects were observed in rats fed diets containing grain from 98140 or 98140+Gly/SU maize with respect to standard nutritional performance metrics and OECD 408-compliant toxicological response variables [OECD, 1998. Section 4 (Part 408), Health Effects: Repeated Dose 90-Day Oral Toxicity Study in Rodents, Guideline for the Testing of Chemicals. Organisation of Economic Co-operation and Development, Paris, France]. These results support the comparative safety and nutritional value of maize grain from genetically modified Optimum® GAT® and conventional, non-transgenic hybrid field corn.

Automatic ontology creation using adaptation

AbstractOntologies are an emerging means of knowledge representation to improve information organization and management, and they are becoming more prevalent in the domain of engineering design. The task of creating new ontologies manually is not only tedious and cumbersome but also time consuming and expensive. Research aimed at addressing these problems in creating ontologies has investigated methods of automating ontology reuse mainly by extracting smaller application ontologies from larger, more general purpose ontologies. Motivated by the wide variety of existing learning algorithms, this paper describes a new approach focused on the reuse of domain-specific ontologies. The approach integrates existing software tools for natural language processing with new algorithms for pruning concepts not relevant to the new domain and extending the pruned ontology by adding relevant concepts. The approach is assessed experimentally by automatically adapting a design rationale ontology for the software engineering domain to a new one for the related domain of engineering design. The experiment produced an ontology that exhibits comparable quality to previous attempts to automate ontology creation as measured by standard content performance metrics such as coverage, accuracy, precision, and recall. However, further analysis of the ontology suggests that the automated approach should be augmented with recommendations presented to a domain expert who monitors the pruning and extending processes in order to improve the structure of the ontology.

Classifying disease outbreak reports using n-grams and semantic features

Introduction This paper explores the benefits of using n-grams and semantic features for the classification of disease outbreak reports, in the context of the BioCaster disease outbreak report text mining system. A novel feature of this work is the use of a general purpose semantic tagger – the USAS tagger – to generate features. Background We outline the application context for this work (the BioCaster epidemiological text mining system), before going on to describe the experimental data used in our classification experiments (the 1000 document BioCaster corpus). Feature sets Three broad groups of features are used in this work: Named Entity based features, n-gram features, and features derived from the USAS semantic tagger. Methodology Three standard machine learning algorithms – Naïve Bayes, the Support Vector Machine algorithm, and the C4.5 decision tree algorithm – were used for classifying experimental data (that is, the BioCaster corpus). Feature selection was performed using the χ 2 feature selection algorithm. Standard text classification performance metrics – Accuracy, Precision, Recall, Specificity and F-score – are reported. Results A feature representation composed of unigrams, bigrams, trigrams and features derived from a semantic tagger, in conjunction with the Naïve Bayes algorithm and feature selection yielded the highest classification accuracy (and F-score). This result was statistically significant compared to a baseline unigram representation and to previous work on the same task. However, it was feature selection rather than semantic tagging that contributed most to the improved performance. Conclusion This study has shown that for the classification of disease outbreak reports, a combination of bag-of-words, n-grams and semantic features, in conjunction with feature selection, increases classification accuracy at a statistically significant level compared to previous work in this domain.