Data Extraction Programs Research Articles

Validity and Reliability Analysis of the PlotDigitizer Software Program for Data Extraction from Single-Case Graphs

Access to raw data of graphs presented in original articles to calculate the effect size of single-case research is a challenge for researchers conducting studies such as meta-analysis. Researchers typically use data extraction software programs to extract raw data from the graphs in articles. In this study, we aimed to analyze the validity and reliability of the PlotDigitizer software program, which is widely used in literature and an alternative to other data extraction programs, on computers with different operating systems. We performed the digitization of 6.846 data points on three different computers using 15 hypothetical graphs with 20 data series and 186 graphs with 242 data series from 29 published articles to accomplish the goal. Besides, using the values we digitized, we recalculated the 23 effect sizes presented in the original articles for validity analysis. Based on our sampling, we calculated intercoder and intracoder Pearson correlation coefficients. The results showed that PlotDigitizer could be an alternative to other programs as it is free and can run on many current and outdated systems, and it is valid and reliable as it is nearly perfect. Based on the obtained results and considering the data extraction process, we presented various recommendations for the researchers that will use the PlotDigitizer program for the quantitative analysis of single-case graphs.

Gneiss: spreadsheet programming using structured web service data

Web services offer a more reliable and efficient way to access online data than scraping web pages. However, interacting with web services to retrieve data often requires people to write a lot of code. Moreover, many web services return data in complex hierarchical structures that make it difficult for people to perform any further data manipulation. We developed Gneiss, a tool that extends the familiar spreadsheet metaphor to support using structured web service data. Gneiss lets users retrieve or stream arbitrary JSON data returned from web services to a spreadsheet using interaction techniques without writing any code. It introduces a novel visualization that represents hierarchies in data using nested spreadsheet cells and allows users to easily reshape and regroup the extracted structured data. Data flow is two-way between the spreadsheet and the web services, enabling people to easily make a new web service call and retrieve new data by modifying spreadsheet cells. We report results form a user study that showed that Gneiss helped spreadsheet users use and analyze structured data more efficiently than Excel and even outperform professional programmers writing code. We further use a set of examples to demonstrate our tool's ability to create reusable data extraction and manipulation programs that work with complex web service data.

Reliability, Validity, and Usability of Data Extraction Programs for Single-Case Research Designs.

Single-case experimental designs (SCEDs) have been increasingly used in recent years to inform the development and validation of effective interventions in the behavioral sciences. An important aspect of this work has been the extension of meta-analytic and other statistical innovations to SCED data. Standard practice within SCED methods is to display data graphically, which requires subsequent users to extract the data, either manually or using data extraction programs. Previous research has examined issues of reliability and validity of data extraction programs in the past, but typically at an aggregate level. Little is known, however, about the coding of individual data points. We focused on four different software programs that can be used for this purpose (i.e., Ungraph, DataThief, WebPlotDigitizer, and XYit), and examined the reliability of numeric coding, the validity compared with real data, and overall program usability. This study indicates that the reliability and validity of the retrieved data are independent of the specific software program, but are dependent on the individual single-case study graphs. Differences were found in program usability in terms of user friendliness, data retrieval time, and license costs. Ungraph and WebPlotDigitizer received the highest usability scores. DataThief was perceived as unacceptable and the time needed to retrieve the data was double that of the other three programs. WebPlotDigitizer was the only program free to use. As a consequence, WebPlotDigitizer turned out to be the best option in terms of usability, time to retrieve the data, and costs, although the usability scores of Ungraph were also strong.

Comparative evaluation of the reliability and validity of three data extraction programs: UnGraph, GraphClick, and DigitizeIt

The use of evidence-based practices in education has been gaining a lot of attention in recent years. Researchers often use meta-analyses to identify evidence-based practices. To conduct meta-analyses of studies employing single-subject experimental research (SSER) designs for the purpose of identifying evidence base for a practice, a necessary step is to obtain raw data from published graphs. One method for obtaining raw data from published SSER graphs is the use of computer programs specifically designed to extract data from graphs. Purpose of the present study was to examine the reliability and validity of three data extraction programs, Ungraph, GraphClick, and DigitizeIt, using 60 graphs obtained from 15 SSER studies focused on a practice. Three coders extracted data from the graphs using the three programs. Values extracted by each coder were compared to (a) each other (reliability) and (b) values reported in the original articles in which the graphs were obtained from (validity). Results showed that raw data from SSER graphs can be obtained reliably using all three data extraction programs and values obtained using the three programs are highly valid. These results suggest that researchers can use data extracted using these programs with a high level of confidences while conducting meta-analyses of studies employing SSER designs. Authors make recommendations for improving the accuracy of data extraction using the three programs.

Clinical data extraction and feedback in general practice: a case study from Australian primary care

Quality improvement in general practice has increasingly focused on the analysis of its clinical databases to guide its improvement strategies. However, general practitioners (GPs) need to be motivated to extract and review their clinical data, and they need skills to do so. This study examines the initial experience of 15 practices in undertaking clinical data extraction and management and the support they were given by their local division of general practice. To explore the uptake of data extraction tools in general practice and understand how divisions of general practice can assist with their uptake. This study was conducted within a single division of general practice within the south-eastern suburbs of metropolitan Melbourne, Australia. Self-selected practices were offered a data extraction program ('tool') free of charge, with ongoing division support. Practice representatives, either GPs, practice nurses or other practice staff members, were given instructions on how to extract data using the data extraction tool. This was followed by discussion with division staff regarding which clinical areas might be focused on. Division staff systematically recorded information about the experience of the practices and collated their clinical data. Fifteen practices, representing 69 GPs, participated. The practices chose from the following areas to work on as quality improvement activities: improving data entry; inactivating patient files for those who no longer attended the practice; correcting demographic information; diabetes and coronary heart disease management. The recording of data, according to the extraction tool, was found to be incomplete. For example, one-third of the patients who had HbA1cs recorded were on target, i.e. <7%, but nearly half the patients with diabetes did not have HbA1cs recorded at all. About half the patients with coronary heart disease were not reported as taking aspirin and one-third were not on a statin. Nearly half the patients who had attended their practice in the previous 30 months did not have smoking status recorded. While data extraction programs provide GPs with useful tools for examining their clinical databases and identifying clinical practice issues which could be improved, external support, such as that provided by divisions, is helpful. Technical barriers, such as the failure of extraction tools to recognise some data and the failure to comprehensively enter data, are impediments, but in spite of these considerable interest exists in the use of clinical data to improve practice.

Structured Data Extraction from the Web Based on Partial Tree Alignment

This paper studies the problem of structured data extraction from arbitrary Web pages. The objective of the proposed research is to automatically segment data records in a page, extract data items/fields from these records, and store the extracted data in a database. Existing methods addressing the problem can be classified into three categories. Methods in the first category provide some languages to facilitate the construction of data extraction systems. Methods in the second category use machine learning techniques to learn wrappers (which are data extraction programs) from human labeled examples. Manual labeling is time-consuming and is hard to scale to a large number of sites on the Web. Methods in the third category are based on the idea of automatic pattern discovery. However, multiple pages that conform to a common schema are usually needed as the input. In this paper, we propose a novel and effective technique (called DEPTA) to perform the task of Web data extraction automatically. The method consists of two steps: 1) identifying individual records in a page and 2) aligning and extracting data items from the identified records. For step 1, a method based on visual information and tree matching is used to segment data records. For step 2, a novel partial alignment technique is proposed. This method aligns only those data items in a pair of records that can be aligned with certainty, making no commitment on the rest of the items. Experimental results obtained using a large number of Web pages from diverse domains show that the proposed two-step technique is highly effective.

Procedure Modeling and Performance Prediction for Navy Surface Ship Consoles

Very little human performance data has been collected for current navy shipboard consoles. This project identified a source of user behavioral data through the reduction and analysis of data captured with the Aegis shipboard data extraction programs. The data was used to construct a preliminary human performance database for a combat team. This database was used as input to a procedural network model, providing an analysis tool which allows task construction and performance prediction without the difficulty associated with acquiring ship or land-based combat system time. The database and models are currently being evaluated for their predictive validity.