Mean False Positive Rate Research Articles

Temporal proteomic analysis of IGF‐1R signalling in MCF‐7 breast adenocarcinoma cells

Dysregulation of the insulin-like growth factor 1 receptor signalling network is implicated in tumour growth and resistance to chemotherapy. We explored proteomic changes resulting from insulin-like growth factor 1 stimulation of MCF-7 adenocarcinoma cells as a function of time. Quantitative analysis using iTRAQ reagents and 2-D LC-MS/MS analysis of three biological replicates resulted in the identification of 899 proteins (p<or=0.05) with an estimated mean false-positive rate of 2.6%. Quantitative protein expression was obtained from 681 proteins. Further analysis by supervised k-means clustering identified five temporal clusters, which were submitted to the FuncAssociate server to assign overrepresented gene ontology terms. Proteins associated with vesicle transport were significantly overrepresented. We further analyzed our data set for proteins showing temporal significance using the software, extraction and analysis of differential gene expression, resulting in 20 significantly and temporally changing proteins (p<or=0.1). These significant proteins play roles in, among others, altered glucose metabolism (lactate dehydrogenase A and pyruvate kinase M1/M2) and cellular stress (nascent polypeptide-associated complex subunit alpha and heat shock (HSC70) proteins). We used multiple reaction monitoring to validate these interesting proteins and have revealed several differences in relative peptide expression corresponding to protein isoforms and variants.

Computer-aided Detection in Computed Tomography Colonography with Full Fecal Tagging: Comparison of Standalone Performance of 3 Automated Polyp Detection Systems

Purpose We sought to compare the performance of 3 computer-aided detection (CAD) polyp algorithms in computed tomography colonography (CTC) with fecal tagging. Methods CTC data sets of 33 patients were retrospectively analysed by 3 different CAD systems: system 1, MedicSight; system 2, Colon CAD; and system 3, Polyp Enhanced View. The polyp database comprised 53 lesions, including 6 cases of colorectal cancer, and was established by consensus reading and comparison with colonoscopy. Lesions ranged from 6-40 mm, with 25 lesions larger than 10 mm in size. Detection and false-positive (FP) rates were calculated. Results CAD systems 1 and 2 could be set to have varying sensitivities with higher FP rates for higher sensitivity levels. Sensitivities for system 1 ranged from 73%–94% for all lesions (78%–100% for lesions ≥10 mm) and, for system 2, from 64%–94% (78%–100% for lesions ≥10 mm). System 3 reached an overall sensitivity of 76% (100% for lesions ≥10 mm). The mean FP rate per patient ranged from 8–32 for system 1, from 1–8 for system 2, and was 5 for system 3. At the highest sensitivity level for all polyps (94%), system 2 showed a statistically significant lower FP rate compared with system 1 ( P = .001). When analysing lesions ≥10 mm, system 3 had significantly fewer FPs than systems 1 and 2 ( P < .012). Conclusions Standalone CTC-CAD analysis in the selected patient collective showed the 3 systems tested to have a variable but overall promising performance with respect to sensitivity and the FP rate.

Analysis of sensorimotor rhythms for the implementation of a brain switch for healthy subjects

This paper presents an asynchronous brain switch using one Laplacian electroencephalographic (EEG) derivation. The brain switch is operated through foot motor imagery (MI) and is based on the classification of event-related desynchronization (ERD) during a motor task or event-related synchronization (ERS) after the termination of the task (also known as the beta rebound). The methods described in this work are suitable for operating a brain–computer interface (BCI) as an attractive control alternative for healthy users. A general description of ERD/ERS is obtained with several band power features and a rigid paradigm timing. Two support vector machines (SVMs) are trained in a novel fashion by using the patterns from motor execution (ME) and a priori information about the significance of ERD/ERS patterns. A maximum true positive rate (TPR) of 0.92 and a minimum of 0.43 was achieved (in 8 out of 9 subjects) during training of the classifiers; with a mean false positive rate (FPR) of 0.09 ± 0.05. A simulation of an asynchronous BCI using MI data and the classifiers trained with ME data achieved a maximum TPR of 0.88, a minimum of 0.50 (in 6 out of 9 subjects) and an average FPR of 0.09 ± 0.04. This work presents a step forward towards an easy-to-set-up and easy-to-use asynchronous BCI system for healthy users.

Computer-Aided Identification of the Pectoral Muscle in Digitized Mammograms

Mammograms are X-ray images of human breast which are normally used to detect breast cancer. The presence of pectoral muscle in mammograms may disturb the detection of breast cancer as the pectoral muscle and mammographic parenchyma appear similar. So, the suppression or exclusion of the pectoral muscle from the mammograms is demanded for computer-aided analysis which requires the identification of the pectoral muscle. The main objective of this study is to propose an automated method to efficiently identify the pectoral muscle in medio-lateral oblique-view mammograms. This method uses a proposed graph cut-based image segmentation technique for identifying the pectoral muscle edge. The identified pectoral muscle edge is found to be ragged. Hence, the pectoral muscle is smoothly represented using Bezier curve which uses the control points obtained from the pectoral muscle edge. The proposed work was tested on a public dataset of medio-lateral oblique-view mammograms obtained from mammographic image analysis society database, and its performance was compared with the state-of-the-art methods reported in the literature. The mean false positive and false negative rates of the proposed method over randomly chosen 84 mammograms were calculated, respectively, as 0.64% and 5.58%. Also, with respect to the number of results with small error, the proposed method out performs existing methods. These results indicate that the proposed method can be used to accurately identify the pectoral muscle on medio-lateral oblique view mammograms.

Computer-aided detection (CAD) as a second reader using perspective filet view at CT colonography: effect on performance of inexperienced readers

To evaluate whether computer-aided detection (CAD) as a second reader using perspective filet view [three-dimensional (3D) filet] improves the performance of inexperienced readers at computed tomography colonography (CTC) compared with unassisted 3D filet and unassisted two-dimensional (2D) CTC. Fifty symptomatic patients underwent CTC and same-day colonoscopy with segmental unblinding. Two inexperienced readers read the CTC studies on 3D filet and 2D several weeks apart. Four months later, readers re-read the cases only evaluating CAD marks using 3D filet. Suspicious CAD marks not previously described on 3D filet were recorded. Jackknife free-response receiver operating characteristic (JAFROC-1) analysis was used to compare the observers' performances in detecting lesions with 3D filet, 2D and 3D filet with CAD. One hundred and three lesions > or =3mm were detected at colonoscopy with segmental unblinding. CAD alone had a sensitivity of 73% (75/103) at a mean false-positive rate per patient of 12.8 in supine and 11.4 in prone. For inexperienced readers sensitivities with 3D filet with CAD were 58% (60/103) and 48% (50/103) with an improvement of 14-16 percentage points (p<0.05) compared with 2D and of 10-11 percentage points (p<0.05) compared with 3D filet. For inexperienced readers, the false-positive rate was 25-41% and 71-200% higher with 3D filet with CAD compared with 3D filet and 2D, respectively. JAFROC-1 analysis showed no significant differences in per-lesion overall performance among reading modes (p=0.8). CAD applied as a second reader using 3D filet increased both sensitivity and the number of false positives by inexperienced readers compared with 3D filet and 2D, thus not improving overall performance, i.e., the ability to distinguish between lesions and non-lesions.

Evaluation of atlas-based segmentation of hippocampi in healthy humans

Region of interest (ROI)-based functional magnetic resonance imaging (fMRI) data analysis relies on extracting signals from a specific area which is presumed to be involved in the brain activity being studied. The hippocampus is of interest in many functional connectivity studies for example in epilepsy as it plays an important role in epileptogenesis. In this context, ROI may be defined using different techniques. Our study aims at evaluating the spatial correspondence of hippocampal ROIs obtained using three brain atlases with hippocampal ROI obtained using an automatic segmentation algorithm dedicated to the hippocampus. High-resolution volumetric T1-weighted MR images of 18 healthy volunteers (five females) were acquired on a 3T scanner. Individual ROIs for both hippocampi of each subject were segmented from the MR images using an automatic hippocampus and amygdala segmentation software called SACHA providing the gold standard ROI for comparison with the atlas-derived results. For each subject, hippocampal ROIs were also obtained using three brain atlases: PickAtlas available as a commonly used software toolbox; automated anatomical labeling (AAL) atlas included as a subset of ROI into PickAtlas toolbox and a frequency-based brain atlas by Hammers et al. The levels of agreement between the SACHA results and those obtained using the atlases were assessed based on quantitative indices measuring volume differences and spatial overlap. The comparison was performed in standard Montreal Neurological Institute space, the registration being obtained with SPM5 (http://www.fil.ion.ucl.ac.uk/spm/). The mean volumetric error across all subjects was 73% for hippocampal ROIs derived from AAL atlas; 20% in case of ROIs derived from the Hammers atlas and 107% for ROIs derived from PickAtlas. The mean false-positive and false-negative classification rates were 60% and 10% respectively for the AAL atlas; 16% and 32% for the Hammers atlas and 6% and 72% for the PickAtlas. Though atlas-based ROI definition may be convenient, the resulting ROIs may be poor representations of the hippocampus in some studies critical to under- or oversampling. Performance of the AAL atlas was inferior to that of the Hammers atlas. Hippocampal ROIs derived from PickAtlas are highly significantly smaller, and this results in the worst performance out of three atlases. It is advisable that the defined ROIs should be verified with knowledge of neuroanatomy before using it for further data analysis.

Assessment of Radiologist Performance in the Detection of Lung Nodules: Dependence on the Definition of “Truth”

Studies that evaluate the lung nodule detection performance of radiologists or computerized methods depend on an initial inventory of the nodules within the thoracic images (the "truth"). The purpose of this study was to analyze (1) variability in the "truth" defined by different combinations of experienced thoracic radiologists and (2) variability in the performance of other experienced thoracic radiologists based on these definitions of "truth" in the context of lung nodule detection in computed tomographic (CT) scans. Twenty-five thoracic CT scans were reviewed by four thoracic radiologists, who independently marked lesions they considered to be nodules >or=3 mm in maximum diameter. Panel "truth" sets of nodules were then derived from the nodules marked by different combinations of two and three of these four radiologists. The nodule detection performance of the other radiologists was evaluated based on these panel "truth" sets. The number of "true" nodules in the different panel "truth" sets ranged from 15 to 89 (mean 49.8 +/- 25.6). The mean radiologist nodule detection sensitivities across radiologists and panel "truth" sets for different panel "truth" conditions ranged from 51.0 to 83.2%; mean false-positive rates ranged from 0.33 to 1.39 per case. Substantial variability exists across radiologists in the task of lung nodule identification in CT scans. The definition of "truth" on which lung nodule detection studies are based must be carefully considered, because even experienced thoracic radiologists may not perform well when measured against the "truth" established by other experienced thoracic radiologists.

Enquête sur la qualité du diagnostic de lèpre à Madagascar

By 2005, Madagascar had not achieved its goal of eliminating leprosy. During reexamination of leprosy patients, rates of diagnostic error ranged from 4.5 to 62%, casting doubt on the reported prevalence of leprosy. We therefore decided to carry out a survey of the quality of leprosy diagnosis. The survey consisted in reexamination of a sample of 102 new cases of leprosy (treated for less than three months). The sample was obtained from clusters of six patients, randomly drawn from the 111 districts in the country's six provinces. Two reexamination teams visited the target districts. Each team included at least three medical doctors: a doctor from the National Program, a WHO consultant and a dermatologist from partner NGOs in the program. The mean false-positive rate was 27.5%, ranging from 5.6 to 44.4% in the different provinces. The quality of leprosy diagnosis was found to be very poor, particularly in districts with a marked decrease in annual detection of leprosy cases. The high rate of false-positives during this survey could be due to the incompetence of peripheral health workers. This incompetence could be partly attributed to the decrease in leprosy detection, resulting in reduced familiarity of these health staff with leprosy diagnosis. Recommendations were made to the country concerning review of the leprosy case detection network and improvement of the quality of leprosy diagnosis in the field. Health workers involved in leprosy detection must have basic dermatological knowledge.

Performance of a Previously Validated CT Colonography Computer-Aided Detection System in a New Patient Population

A computer-aided detection (CAD) system with high sensitivity in the detection of adenomatous polyps in varied CT colonography (CTC) data sets increases the utility of CAD in the clinical setting. The purpose of this study was to evaluate the standalone performance of an existing CAD system with a new set of CTC data from screening patients at an institution and geographic location different from those at which the CAD system was trained. CTC data were collected from the records of 104 patients undergoing screening for colorectal neoplasia. Most of the patients were at average risk, had CTC findings suggestive of polyps, and underwent colonoscopy. Patients underwent cathartic bowel preparation, were given an oral contrast agent, and underwent imaging in the prone and supine positions. The patients had 86 adenomas confirmed at same-day optical colonoscopy; 47 of these tumors were 10 mm in diameter or larger, and 39 measured 6-9 mm. The CTC data were analyzed with an existing CAD system for colonography that was trained with previously acquired data. In a previous non-polyp-enriched screening cohort, the standalone performance of the CAD system was 93.3% (28/30) sensitivity for adenomatous polyps 10 mm or larger, 51.1% (47/92) sensitivity for adenomas 6-9 mm, and a mean false-positive rate of 8.6 per patient. Sensitivity comparisons were made with findings in the previous study. The CAD system had per-polyp sensitivities of 91.5% (43/47; 95% CI, 78.7-97.2%; p = 1.0) for adenomas 10 mm or larger and 82.1% (32/39; 65.9-91.9%; p = 0.0009) for adenomas 6-9 mm. The per-patient sensitivities were 97.6% (40/41; 85.6-99.9%; p = 0.6) for patients with adenomas 10 mm or larger and 82.4% (28/34; 64.8-92.6%; p = 0.047) for patients with adenomas 6-9 mm. The mean and median false-positive rates were 9.6 +/- 9.6 and 7.0 per patient, respectively. Common reasons for CAD misses (false-negative findings) were the presence of adherent contrast medium, flat adenomas, and adenomas located on or adjacent to normal colonic folds. In a random sample, 72.5% (29/40) of false-positive findings were attributable to folds or residual feces. The CAD system evaluated has a high level of performance in the detection of adenomatous polyps with CTC data from a polyp-enriched cohort different from that used to train the system.

Automatic analysis of immunocytochemically stained tissue samples

An automatic colour image segmentation and cell counting software system has been developed for immunocytochemical analysis of stained tissue samples. The system was designed to count the total number of positive and negative cells in tissue samples treated with cytokine DNA probes from pigs naturally parasitised with Taenia solium metacestodes, using in situ hybridisation. A reaction index was calculated as the ratio of the number of cells with a positive reaction to the total number of cells (positives plus negatives) for each of five different probes. The objectives of automatic counting were to improve the reproducibility of the analysis and reduce the processing time of large image batches. A fast KNN classifier was used for colour segmentation. Watershed segmentation combined with edge detection was used to isolate individual cells that were then automatically labelled, using the results of the corresponding colour segmented image. Validation was performed on 122 non-training digital images with a total of 1069 positive cells and 1459 negative cells, with the following results: a mean true positive rate of 90.2% for positive cells and a mean true positive rate of 85.4% for negative cells. The corresponding mean false positive rates were 9.6% and 6.6%. The mean reaction index error of the automatic analysis was 5.35%. The processing of each digital image took 10 s on a Pentium IV PC.

Physician Predictors of Mammographic Accuracy

The association between physician experience and the accuracy of screening mammography in community practice is not well studied. We identified characteristics of U.S. physicians associated with the accuracy of screening mammography. Data were obtained from the Breast Cancer Surveillance Consortium and the American Medical Association Master File. Unadjusted mammography sensitivity and specificity were calculated according to physician characteristics. We modeled mammography sensitivity and specificity by multivariable logistic regression as a function of patient and physician characteristics. All statistical tests were two-sided. We studied 209 physicians who interpreted 1,220,046 screening mammograms from January 1, 1995, through December 31, 2000, of which 7143 (5.9 per 1000 mammograms) were associated with breast cancer within 12 months of screening. Each physician interpreted a mean of 6011 screening mammograms (95% confidence interval [CI] = 4998 to 6677), including a mean of 34 (95% CI = 28 to 40) from women diagnosed with breast cancer. The mean sensitivity was 77% (range = 29%-97%), and the mean false-positive rate was 10% (range = 1%-29%). After adjustment for the patient characteristics of those whose mammograms they interpreted, physician characteristics were strongly associated with specificity. Higher specificity was associated with at least 25 years (versus less than 10 years) since receipt of a medical degree (for physicians practicing for 25-29 years, odds ratio [OR] = 1.54, 95% CI = 1.14 to 2.08; P = .006), interpretation of 2500-4000 (versus 481-750) screening mammograms annually (OR = 1.30, 95% CI = 1.06 to 1.59; P = .011) and a high focus on screening mammography compared with diagnostic mammography (OR = 1.59, 95% CI = 1.37 to 1.82; P<.001). Higher overall accuracy was associated with more experience and with a higher focus on screening mammography. Compared with physicians who interpret 481-750 mammograms annually and had a low screening focus, physicians who interpret 2500-4000 mammograms annually and had a high screening focus had approximately 50% fewer false-positive examinations and detected a few less cancers. Raising the annual volume requirements in the Mammography Quality Standards Act might improve the overall quality of screening mammography in the United States.

Clinical evaluation of wavelet-compressed digitized screen-film mammography

The authors compared diagnostic accuracy and callback rates with conventional screen-film mammograms and wavelet-compressed digitized images. Sixty sets of mammograms (four views per case) were digitized at a spatial resolution of 100 microm. The images were wavelet compressed to a mean compression ratio of 8:1 and reviewed by three mammographers. Five regions were evaluated in each breast. Suspicion of malignancy was graded on a scale of 0% to 100%, and receiver operating characteristic (ROC) analysis was performed. Callback rates were calculated by using the American College of Radiology's Breast Imaging Reporting and Data System lexicon scale. The mean diagnostic accuracy with compressed and conventional images was 0.832 and 0.860, respectively. The upper 95% confidence bound for the difference in ROC areas was 0.061. The mean false-positive rate at a fixed sensitivity of 0.90 was 0.041 for compressed images and 0.059 for conventional images. The mean callback rates for normal, benign, and malignant regions were 0.023, 0.305, and 0.677, respectively, for compressed images and 0.036, 0.447, and 0.750, respectively, for conventional images. The upper 95% confidence bound for the (absolute) differences in callback rates was 0.012 for normal regions, 0.163 for benign regions, and 0.138 for malignant regions. Diagnostic accuracies were equivalent for both compressed and conventional images. The mean false-positive rate at fixed sensitivity was much better with the compressed images. However, the callback rates for malignant lesions were lower when the compressed images were used.

Automated detection of spleen volume by spiral CT scans using neural networks and "fuzzy logic"

To assess spleen segmentation and volumentry in spiral CT scans with and without pathological changes of splenic tissue. The image analysis software HYBRIKON is based on region growing, self-organized neural nets, and fuzzy-anatomic rules. The neural nets were trained with spiral CT data from 10 patients, not used in the following evaluation on spiral CT scans from 19 patients. An experienced radiologist verified the results. The true positive and false positive areas were compared in terms to the areas marked by the radiologist. The results were compared with a standard thresholding method. The neural nets achieved a higher accuracy than the thresholding method. Correlation coefficient of the fuzzy-neural nets: 0.99 (thresholding: 0.63). Mean true positive rate: 90% (thresholding: 75%), mean false positive rate: 5% (thresholding > 100%). Pitfalls were caused by accessory spleens, extreme changes in the morphology (tumors, metastases, cysts), and parasplenic masses. Self-organizing neural nets combined with fuzzy rules are ready for use in the automatic detection and volumetry of the spleen in spiral CT scans.

Accuracy of administrative data for assessing outcomes after knee replacement surgery

Objective: To assess the accuracy of information in an administrative database (Canadian Institute for Health Information; CIHI) compared with the hospital record for patients undergoing knee replacement (KR). Methods: A stratified random sample of 185 KR recipients from 5 Ontario hospitals were chosen. Their hospital records and corresponding CIHI files were compared to assess percent complete agreement, false negative (FN) and false positive (FP) rates for demographic data, procedures, and diagnoses. Results: Of 185 records, 175 (95%) were reviewed. Percent complete agreement was greater than 94% for each of patient demographics and procedures (mean FN rates: 0%; mean FP rates: 0–5%). For comorbidities and complications, although mean percent complete agreement was high, and FP rates were low, mean FN rates were 63% for specific comorbid conditions and 70% for organ systems. Conclusions: High FN rates have been found in documentation of comorbidities and in-hospital complications for CIHI data compared with the hospital record. Under-coding of comorbidities and in-hospital complications has potential implications for researchers using administrative databases.