Human Readers Research Articles

Feasibility of Automated Quantification of Regional Disease Patterns Depicted on High-Resolution Computed Tomography in Patients with Various Diffuse Lung Diseases

ObjectiveThis study was designed to develop an automated system for quantification of various regional disease patterns of diffuse lung diseases as depicted on high-resolution computed tomography (HRCT) and to compare the performance of the automated system with human readers.Materials and MethodsA total of 600 circular regions-of-interest (ROIs), 10 pixels in diameter, were utilized. The 600 ROIs comprised 100 ROIs that represented six typical regional patterns (normal, ground-glass opacity, reticular opacity, honeycombing, emphysema, and consolidation). The ROIs were used to train the automated classification system based on the use of a Support Vector Machine classifier and 37 features of texture and shape. The performance of the classification system was tested with a 5-fold cross-validation method. An automated quantification system was developed with a moving ROI in the lung area, which helped classify each pixel into six categories. A total of 92 HRCT images obtained from patients with different diseases were used to validate the quantification system. Two radiologists independently classified lung areas of the same CT images into six patterns using the manual drawing function of dedicated software. Agreement between the automated system and the readers and between the two individual readers was assessed.ResultsThe overall accuracy of the system to classify each disease pattern based on the typical ROIs was 89%. When the quantification results were examined, the average agreement between the system and each radiologist was 52% and 49%, respectively. The agreement between the two radiologists was 67%.ConclusionAn automated quantification system for various regional patterns of diffuse interstitial lung diseases can be used for objective and reproducible assessment of disease severity.

Pages in peril: what we may lose as ebooks go mainstream

The ebook industry is growing asymmetrically: Far more novels are being read than nonfiction texts, and those are generally read as reflowable text on small devices. Uniform pagination is seen as unnecessary, raising the risk that ebook technology will mature without a standard mechanism for citing pages in research footnotes and discussion, making traditional source attribution of older works impossible. The author suggests a publisher-side ‘master view’ of books including both print-image pagination and matching page markers embedded in reflowable ebooks. Appropriate ebook rendering software would allow human readers to jump to ebook page locations identical to those in print editions of the same work. ‘Decimal’ page references (such as p. 206.339) could allow precise citation of sentences or even individual words in an ebook without sacrificing human comprehension of pagination in printed works. Such a system would preserve page-based footnoting in both directions between existing print books and ebooks.

Location specific radiographic joint space width for osteoarthritis progression

To establish the performance of location specific computer measures of radiographic joint space width (JSW) compared to measurements of minimum joint space width (mJSW) for the assessment of medial compartment knee osteoarthritis (OA). The study also investigated the most disease-responsive location for measuring medial compartment JSW. Serial bilateral Posterior Anterior (PA) conventional radiographs acquired with a fixed flexion protocol were obtained 36 months apart in 118 persons with knee OA participating in the Health, Aging and Body Composition (Health ABC) Study. Measurements of medial compartment mJSW and JSW at seven fixed locations were facilitated by the use of semi-automated software that delineated the femoral and tibial margins of the joint. A human reader operated custom software to verify and correct the software-drawn margins where necessary. Paired images were displayed with the reader blinded to the chronological order. The amount of joint space narrowing was measured and the standardized response mean (SRM) was used as a metric to quantify performance. For all subjects, the mJSW SRM value was 0.42 while, for the most responsive location specific measure of JSW, it was SRM=0.46. For subjects with a Kellgren-Lawrence (KL) score less than or equal to 1, mJSW (SRM=0.40) was more responsive than the new measures (Maximum SRM=0.30). For KL=2or3, SRM=0.49 for mJSW, and SRM=0.74 for the most responsive location specific measure of JSW. Improved responsiveness was observed in the more central portion of the joint on the more diseased knees. Location specific computer measures of JSW are feasible and potentially provide a superior method to assess radiographic OA for more diseased subjects. This new measure has the potential to improve the power of clinical studies that use a fixed flexion protocol.

389 STUDY OF LOCATION SPECIFIC LATERAL COMPARTMENT RADIOGRAPHIC JOINT SPACE WIDTH FOR KNEE OSTEOARTHRITIS PROGRESSION: ANALYSIS OF LONGITUDINAL DATA FROM THE OSTEOARTHRITIS INITIATIVE (OAI)

Purpose: To study the performance of location-specific radiographic joint space width (JSW) in the lateral compartment. Lateral compartment JSW is a potentially important measurement for subjects with lateral compartment OA, and as a measure of pseudowidening for medial compartment disease. Methods: Baseline and Year 1 knee radiographs of 50 subjects from the Progression Cohort of the Osteoarthritis Initiative (OAI) were analyzed using a software technique that measured the radiographic joint space width (JSW). The data were a subset of OAI Image Releases 0.1.1, 0.B.1, and 1.B.1. Radiographic JSW and the baseline varus-valgus anatomical alignment angle was measured on a single indexed knee for each subject. Measurements of lateral and medial compartment JSW at fixed locations were facilitated by the use of automated software that delineated the femoral and tibial margins of the joint. Measures of JSW were defined as the distance from the tibial margin to the femur margin at fixed locations on the coordinate system shown in Figure 1. JSW was measured at nine fixed locations (x = 0.7, x = 0.725, x = 0.75, x = 0.775 x = 0.8, x = 0.825, x = 0.85, x = 0.875, and x = 0.9) A subset of 22 subjects were defined as having lateral compartment OA based on an anatomical angle of greater than 3 degrees valgus. Lateral compartment minimum JSW (mJSW) was measured for this subset. A human reader operated custom software to verify and correct the software-drawn margins where necessary. Paired images were displayed with the reader blinded to the time point. The average and standard deviation of the JSW loss, and the standardized response means (SRMs) are reported. Results: Table 1 provides measurements of JSW gain for the 28 subjects defined as having medial compartment OA. Increased JSW is observed for locations in the outer portion of the joint (higher x value). Table 2 gives the results for the 22 subjects defined as having lateral compartment OA. Here, decreased JSW is evident for locations in the more central portion of the joint (lower x value). To investigate the lateral compartment widening for medial compartment OA, we examined the correlation between JSW (x = 0.2) and JSW (x = 0.8), for the 28 subjects with an anatomical angle 3 degrees. A modest negative correlation was observed (R=−0.20, p = 0.15, Figure 2).

Two systematic reviews to compare effects of double reading and computer-aided detection on both cancer detection and recall rate

There are two competing methods for improving the accuracy of a single screening radiologist: use of a computer aid (CAD) or double reading. Bibliographic databases were searched for studies where either intervention was incorporated into routine screening work. Meta-analyses were performed to find overall estimates of the impacts of CAD and double reading on both the cancer detection rate and the recall rate. Ten studies were found comparing single reading with CAD to single reading. Seventeen studies were found comparing double reading to single reading. Double reading generally increases the cancer detection rate, but also the recall rate. However, double reading with arbitration increases the detection rate (95% CI = 1.02 to 1.15) and decreases the recall rate (95% CI = 0.92 to 0.96). CAD does not have a significant effect on the cancer detection rate (95% CI = 0.96 to 1.13) and increases the recall rate (95% CI = 1.09 to 1.12). However, there is considerable heterogeneity in the impact on the recall rate in both sets of studies. There is better evidence for an improvement in the cancer detection rate with a human second reader than with CAD. Arbitration where two readers disagree also delivers a reduced recall rate, whereas CAD increases the recall rate. There are therefore strong grounds for preferring double reading with arbitration to single reading with CAD.

An improved algorithm for femoropopliteal artery centerline restoration using prior knowledge of shapes and image space data

Accurate arterial centerline extraction is essential for comprehensive visualization in CT Angiography. Time consuming manual tracking is needed when automated methods fail to track centerlines through severely diseased and occluded vessels. A previously described algorithm, Partial Vector Space Projection (PVSP), which uses vessel shape information from a database to bridge occlusions of the femoropopliteal artery, has a limited accuracy in long (>100 mm) occlusions. In this article we introduce a new algorithm, Intermediate Point Detection (IPD), which uses calcifications in the occluded artery to provide additional information about the location of the centerline to facilitate improvement in PVSP performance. It identifies calcified plaque in image space to find the most useful point within the occlusion to improve the estimate from PVSP. In this algorithm candidates for calcified plaque are automatically identified on axial CT slices in a restricted region around the estimate obtained from PVSP. A modified Canny edge detector identifies the edge of the calcified plaque and a convex polygon fit is used to find the edge of the calcification bordering the wall of the vessel. The Hough transform for circles estimates the center of the vessel on the slice, which serves as a candidate intermediate point. Each candidate is characterized by two scores based on radius and relative position within the occluded segment, and a polynomial function is constructed to define a net score representing the potential benefit of using this candidate for improving the centerline. We tested our approach in 44 femoropopliteal artery occlusions of lengths up to 398 mm in 30 patients with peripheral arterial occlusive disease. Centerlines were tracked manually by four-experts, twice each, with their mean serving as the reference standard. All occlusions were first interpolated with PVSP using a database of femoropopliteal arterial shapes obtained from a total of 60 subjects. Occlusions longer than 80 mm (N = 20) were then processed with the IPD algorithm, provided calcifications were found (N = 14). We used the maximum point-wise distance of an interpolated curve from the reference standard as our error metric. The IPD algorithm significantly reduced the average error of the initial PVSP from 2.76 to 1.86 mm (p < 0.01). The error was less than the clinically desirable 3 mm (smallest radius of the femoropopliteal artery) in 13 of 14 occlusions. The IPD algorithm achieved results within the range of the human readers in 11 of 14 cases. We conclude that the additional use of sparse but specific image space information, such as calcified atherosclerotic plaque, can be used to substantially improve the performance of a previously described knowledge-based method to restore the centerlines of femoropopliteal arterial occlusions.

Converting Taxonomic Descriptions to New Digital Formats

Abstract.--The majority of taxonomic descriptions is currently in print format. The majority of digital descriptions are in formats such as DOC, HTML, or PDF and for human readers. These formats do not convey rich semantics in taxonomic descriptions for computer-aided process. Newer digital formats such as XML and RDF accommodate semantic annotations that allow computers to process the rich semantics on human's behalf, thus open up opportunities for a wide range of innovative usages of taxonomic descriptions, such as searching in more precise and flexible ways, integrating with gnomic and geographic information, generating taxonomic keys automatically, and text data mining and information visualization etc. This paper discusses the challenges in automated conversion of multiple collections of descriptions to XML format and reports an automated system, MARTT. MARTT is a machine-learning system that makes use of training examples to tag new descriptions into XML format. A number of utilities are implemented as solutions to the challenges. The utilities are used to reduce the effort for training example preparation, to facilitate the creation of a comprehensive schema, and to predict system performance on a new collection of descriptions. The system has been tested with several plant and alga taxonomic publications including Flora of China and Flora of North America.

280 Sensitivity Comparison Between Human Readers and Computer-Aided Diagnosis Software in the Detection of Polyps in Large-Scale Multicenter Clinical Trials On Virtual Colonoscopy

280 Sensitivity Comparison Between Human Readers and Computer-Aided Diagnosis Software in the Detection of Polyps in Large-Scale Multicenter Clinical Trials On Virtual Colonoscopy

Graphical representation standards for chemical structure diagrams (IUPAC Recommendations 2008)

Abstract The purpose of a chemical structure diagram is to convey information - typically the identity of a molecule - to another human reader or as input to a computer program. Any form of communication, however, requires that all participants understand each other. Recommendations are provided for the display of two-dimensional chemical structure diagrams in ways that avoid ambiguity and are likely to be understood correctly by all viewers. Examples are provided in many areas, ranging from issues of typography and color selection to the relative positioning of portions of a diagram and the rotational alignment of the diagram as a whole. Explanations describe which styles are preferred and which should be avoided. Principal recommendations include: 1) Know your audience: Diagrams that have a wide audience should be drawn as simply as possible; 2) Avoid ambiguous drawing styles; 3) Avoid inconsistent drawing styles.

Reader studies for validation of CAD systems

Evaluation of computational intelligence (CI) systems designed to improve the performance of a human operator is complicated by the need to include the effect of human variability. In this paper we consider human (reader) variability in the context of medical imaging computer-assisted diagnosis (CAD) systems, and we outline how to compare the detection performance of readers with and without the CAD. An effective and statistically powerful comparison can be accomplished with a receiver operating characteristic (ROC) experiment, summarized by the reader-averaged area under the ROC curve (AUC). The comparison requires sophisticated yet well-developed methods for multi-reader multi-case (MRMC) variance analysis. MRMC variance analysis accounts for random readers, random cases, and correlations in the experiment. In this paper, we extend the methods available for estimating this variability. Specifically, we present a method that can treat arbitrary study designs. Most methods treat only the fully-crossed study design, where every reader reads every case in two experimental conditions. We demonstrate our method with a computer simulation, and we assess the statistical power of a variety of study designs.

Blinded Comparison of Computer-Aided Detection with Human Second Reading in Screening Mammography

The purpose of this study was to compare a human second reader with computer-aided detection (CAD) for the reduction of false-negative cases by a primary radiologist. We retrospectively reviewed our clinical practice. We found that 6,381 consecutive screening mammograms were interpreted by a primary reader. This radiologist then reinterpreted the studies using CAD ("CAD reader"). A second human reader who was blinded to the CAD results but knowledgeable of the primary reader's findings reviewed the studies, looking for abnormalities not seen by the first reader. Two cancers were called back by the second human reader that were not called back by the CAD reader; however, the CAD system had marked the findings, but they were dismissed by the primary reader. Because of the small numbers, the difference between the CAD and second human reader was not statistically significant. The CAD and human second readers increased the recall rates 6.4% and 7.2% (p = 0.70), respectively, and the biopsy rates 10% and 14.7%. The positive predictive value was 0% (0/3) for the CAD reader and was 40% (2/5) for the human second reader. The relative increases in the cancer detection rate compared with the primary reader's detection rate were 0% for the CAD reader and 15.4% (2/13) for the human second reader (p = 0.50). A human second reader or the use of a CAD system can increase the cancer detection rate, but we found no statistical difference between the two because of the small sample size. A possible benefit from a human second reader is that CAD systems can only point to possible abnormalities, whereas a human must determine the significance of the finding. Having two humans review a study may increase detection rates due to interpreter--hence, perceptual--variability and not just increased detection.

Blinded Comparison of Computer-Aided Detection with Human Second Reading in Screening Mammography: The Importance of the Question and the Critical Numbers Game

1Department of Radiology, George Washington University, 2150 Pennsylvania Ave., NW, Washington, DC 20037. Address correspondence to R. F. Brem. he authors of the well-designed study reported in the previous article [1] set out to compare the use of computer-aided detection (CAD) and a single radiologist interpretation of screening mammography with double reading for the improved diagnosis of breast cancer. With the increasing use of CAD, as well as the ongoing controversy about its use, this is an important goal and the authors ask a critically important question. However, it is also a challenging goal. The comparison of single reading with CAD versus double reading requires a large patient population to achieve sufficient statistical power to definitively answer the question posed. The admirable study, reported in this issue of AJR [1], is the basis on which additional studies will continue to build our knowledge base of this rapidly expanding field. Although not the primary goal, additional and excellent insights about the use of CAD as well as its integration into clinical practice result from this study. The authors [1] have used sophisticated and extensive statistics. They report the recall rate, biopsy rate, positive predictive value, and increased cancer detection rates for a single reader with CAD as compared with the double reading of mammograms. In this study, 6,381 consecutive screening mammograms were included in which a total of 18 cancers were present, 15 diagnosed and three false-negative cancers that were found on imaging 1 year later. The literature reports between 3 and 8 cancers per 1,000 screening mammograms [2], although the prevalence of breast cancer in the population reported in the study being discussed is 2.03 per thousand for the primary reader and 2.35 for all three “readers,”—that is, the primary reader, the primary reader with CAD, or the primary reader along with the second human reader [1]. The authors ascribe this difference to the fact that these are predominantly incidence cancers and not prevalence cancers because most patients had prior mammograms and suggest that this difference likely reflects the excellent care afforded patients seen in the investigators’ clinical practice. Nevertheless, the result of the study of the 6,381 screening mammograms is 18 cancers. This is the primary limitation of this study. The authors note that the small study size does not allow for a statistical comparison. In fact, they state that “.... the number of cases was too small to determine if the CAD or if the human reader was superior,....” and therefore the question that this study set out to investigate could not be definitely answered. The authors understand the limitations of their study; they state that:

Evaluation of high‐resolution and mobile display systems for digital radiology in dark and bright environments using human and computational observers

Abstract— As digital display systems replace film traditionally used for reading radiographic images, resource‐intensive acceptance testing must be performed to ensure that quality meets and maintains desired specifications. If machine observers can replace human readers, whose performances are highly variable, the results will be more consistent and less costly. To be effective, however, the automated observers must track human performance. An approach for a model observer, validated with human readers, for the evaluation of the visibility of low‐contrast small targets in high‐resolution and mobile displays under different ambient illumination, will be described. The displays were tested using CDMAM‐like digital phantoms containing disks of varying diameters and contrasts on a flat background. For this task, we find the best indicator of display performance to be the display's ability to represent small luminance contrast, not resolution or pixel size. The results confirm that high‐resolution systems perform better under low illumination while illuminance has a minor impact on the mobile‐display performance. Finally, the results show that the machine observer tracks the performance of human readers. Machine observers with proper validation can replace humans in the acceptance testing procedures, saving the testers both time and money.

Comparison of novel computer detectors and human performance for spike detection in intracranial EEG

Objective Interictal spikes in intracranial EEG (iEEG) may correlate with epileptogenic cortex, but review of interictal iEEG is labor intensive. Accurate automated spike detectors are necessary for understanding the role of spikes in epileptogenesis. Methods The sensitivity, accuracy and reproducibility of three automated iEEG spike detectors were compared against two human EEG readers using iEEG segments from eight patients. A consensus set of detections was generated for detector calibration. Spike verification was calculated after both human EEG readers independently reviewed all detections. Results Humans and two of the three automated detectors demonstrated comparable accuracy. In four patients, automated spike detection sensitivity was >70% and accuracy was >50%. In the remaining four patients, EEG background morphology resulted in poorer performance. Blinded human verification accuracy was 76.7 ± 6.6% for computer-detected spikes, and 84.5 ± 4.1% for human-detected spikes. Conclusions Automated iEEG spike detectors perform comparably to humans, but sensitivity and accuracy are patient dependent. Humans verified the majority of computer-detected spikes. Significance In some patients automated detectors may be used for mapping spike occurrences in epileptic networks. This may reveal associations between spike distribution, seizure onset, and pathology.

Towards content-oriented patent document processing

In this article, we present ongoing work on an advanced patent processing service PATExpert. The central assumption underlying PATExpert is that in order to meet the needs of the users of patent processing services, recourse must be made to the content of patent material. We introduce a content representation schema for patent documentation and sketch the design of techniques that facilitate the integration of this schema into the patent processing cycle. Two types of techniques are discussed. Techniques of the first type facilitate the access to the content of patent documentation provided in a textual format – be it by the human reader or by the machine – in that they rephrase and summarize the documentation and map it onto a formal semantic representation. Techniques of the second type operate on the content representation. At this stage, PATExpert is explored in two technology areas – optical recording devices and machine tools. The work is being carried out in the framework of an R&D-project partially funded by the European Commission.

Automated assessment of myocardial SPECT perfusion scintigraphy: A comparison of different approaches of case-based reasoning

This study compared the diagnostic accuracy of different approaches of case-based reasoning (CBR) for the assessment of coronary artery disease (CAD) using thallium-201 myocardial perfusion scintigraphy in comparison with coronary angiography. For each scintigraphic image set, regional myocardial tracer uptake was obtained by polar map analysis. CBR algorithms based on a similarity measure were employed to identify similar scintigraphic images within the case library, where each case contained the scintigraphic data together with results of coronary angiography. The angiographic data of retrieved cases were then used to determine whether significant CAD was present in one of the major coronary arteries. Three different approaches of CBR were compared: (1) case retrieval based on a global comparison of polar map data (GLOB), (2) case retrieval based on a territorial comparison of polar map data (TER), and (3) case retrieval based on a comparison of a given case with eight sub-libraries classified according to the involvement of the three major coronary vessels using a group similarity measure (GROUP). Two matching algorithms the best-match approach and an adapted retrieving approach were combined with all three case retrieval methods and their influence on the diagnostic accuracy were investigated. For overall detection of significant CAD, the best-match approach of both TER and GROUP retrieval methods showed a higher diagnostic accuracy than the GLOB retrieval method (75% and 77% versus 70%, respectively). ROC analysis for the adapted retrieving approach showed a similar diagnostic accuracy for all three methods with an area under the curve of 0.79, 0.8, and 0.8 for GLOB, TER, and GROUP, respectively. The observed improvement in the diagnostic accuracy by the new approaches may lead to further improvements of CBR systems, which have the potential to offer valuable decision support for human readers, especially for less experienced investigators.

Older versions of the ROUGEeval summarization evaluation system were easier to fool

We show some limitations of the ROUGE evaluation method for automatic summarization. We present a method for automatic summarization based on a Markov model of the source text. By a simple greedy word selection strategy, summaries with high ROUGE-scores are generated. These summaries would however not be considered good by human readers. The method can be adapted to trick different settings of the ROUGEeval package.

GridIMAGE: A Novel Use of Grid Computing to Support Interactive Human and Computer-Assisted Detection Decision Support

This paper describes a Grid-aware image reviewing system (GridIMAGE) that allows practitioners to (a) select images from multiple geographically distributed digital imaging and communication in medicine (DICOM) servers, (b) send those images to a specified group of human readers and computer-assisted detection (CAD) algorithms, and (c) obtain and compare interpretations from human readers and CAD algorithms. The currently implemented system was developed using the National Cancer Institute caGrid infrastructure and is designed to support the identification of lung nodules on thoracic computed tomography. However, the infrastructure is general and can support any type of distributed review. caGrid data and analytical services are used to link DICOM image databases and CAD systems and to interact with human readers. Moreover, the service-oriented and distributed structure of the GridIMAGE framework enables a flexible system, which can be deployed in an institution (linking multiple DICOM servers and CAD algorithms) and in a Grid environment (linking the resources of collaborating research groups). GridIMAGE provides a framework that allows practitioners to obtain interpretations from one or more human readers or CAD algorithms. It also provides a mechanism to allow cooperative imaging groups to systematically perform image interpretation tasks associated with research protocols.

Digital Watermarking Using Clear Toner

It is often desirable to protect a document against unauthorized copying. This could be easily achieved by printing part of the image such that it can be readily observed by a human reader but not by a copier or scanner. An image can be printed using clear toner or ink so as to create a difference in reflected and diffused light that can be distinguished by a human reader who looks at the paper at an angle; however, this feature cannot be detected by copiers or scanners because they are restricted to reading at right angles to the page.In the printing industry, clear or white toners are often used to produce digital watermarks that cannot be copied or scanned but can be observed by a human reader. These methods need specifically matched paper and toners that have a considerable difference in gloss. It works with the combination of glossy toner on matte paper or matte toner on glossy paper. However, this approach does not work when glossy paper is used with a toner that provides images in the same gloss range. Such a combination is very common in the field of digital commercial printing.Several digital watermarking processes have been developed that embed a visually striking gloss image into one to four color images. In most cases, controlled angular dependence of image gloss is used to create visual differential gloss in the form of a digital watermark.Printing equipment with fifth imaging station allows for the production of digital watermarks with clear or low-pigmented toner. It provides a method to produce digital watermarks on paper without the limitations described above. This approach is not limited by the color or gloss properties of the toner, the substrate, and/or the selected fusing technology.This technology allows for the production of clear and visible watermarks using toners that produce an image of about the same gloss as the paper and image with the appropriate fusing technology.The additional clear toning station allows, as well, the addition of clear toner to areas of the digital watermarking process that alone cannot provide a sufficient digital watermarking effect in low-density and high-density areas.

Speech synthesis with prosodic phrase boundary information

Text-to-speech conversion uses pattern-matching to predict the position of phrase boundaries in spoken output. Text input to the is analyzed to identify groups of words (known as “chunks”) which are unlikely to contain internal phrase boundaries. Both the chunks and individual words are labeled with their syntactic characteristics. Access is made to a database of sentences which also contains such syntactic labels, together with indications of where a human reader would insert minor and major phrase boundaries. The parts of the database which have the most similar syntactic characteristics are found and phrase boundaries are predicted based on the phrase boundaries found in those parts. Other characteristics may also be used in the pattern-matching process.