Improve Accuracy Of Interpretation Research Articles

Abstract PO4-26-09: The Advantage of Artificial-Intelligence in HER2 IHC 0 and 1+ Scoring in Breast Cancer

Abstract Background: Trastuzumab deruxtecan has substantially changed the treatment of HER2-low breast cancer, emphasizing the need for accurate differentiation between HER2 immunohistochemistry (IHC) scores of 0 and 1+. However, the current accuracy of HER2 IHC 0 and 1+ scoring in real-world is inadequate. Artificial intelligence (AI) has emerged as a potential solution to improve interpretation accuracy. We developed an AI algorithm based on whole slide images (WSI) to quantitatively assess HER2 expression and its role in interpreting HER2 IHC scores of 0 and 1+. Methods: Our three-phase AI analysis framework involved segmenting tumor areas (excluding ductal carcinoma in situ), detecting and classifying tumor cells based on membrane staining patterns, and grading HER2 IHC scores according to the 2018 ASCO/CAP HER2 guideline. The AI tool was trained using 6012 patches annotated by experienced pathologists. Performance evaluation was conducted using a test dataset comprising 265 slides. A total of 141 HER2 IHC slides (27 IHC 0 and 114 IHC 1+) from patients diagnosed with invasive breast cancer at Fudan University Shanghai Cancer Center in 2021 were included. Two pretrained expert pathologists independently rescored the HER2 slides, followed by analysis using the AI tool. All inconsistent cases were also reviewed by a third senior pathologist. Interobserver agreement between the pathologists and concordance between the pathologists and the AI interpretation results were assessed. We also explored potential ranges where HER2 interpretation by pathologists exhibited inconsistency or inaccuracy. Results: The HER2 AI algorithm showed high performance in interpreting all levels of HER2 expression, with an overall kappa value of 0.85. For HER2 IHC 0 and 1+ cases, the AI model demonstrated high sensitivities (0.839 and 0.914) and specificities (0.983 and 0.890) (Table 1). After the rescoring process, 51 cases were reclassified as IHC 0 and 90 cases as IHC 1+. The overall agreement rate between the historical and re-scoring results was 73.05% (103/141). The agreement rate between the AI and re-scoring results was 91.49% (129/141) (Table 2), indicating comparable interpretation capabilities between the AI model and well-trained pathologists. The interobserver agreement between the two pathologists was 83.69% (118/141), with agreement rates of 88.23% (45/51) for IHC 0 and 81.11% (73/90) for IHC 1+. Analyzing the inconsistent cases (n=12) between IHC 0 and IHC 1+, we examined the percentage of weak and incomplete expression provided by the AI. In 9 out of 12 cases, the AI provided the same results as the third senior pathologist, suggesting that AI could assist in interpretation within this range. We identified a range of 2.83% to 24.98% as a "grey-zone," where even well-trained pathologists provided inconsistent results. Conclusion: Our study demonstrates the excellent performance of an AI-based tool for scoring HER2 IHC 0 and 1+. The AI model exhibits comparable capabilities to well-trained pathologists in interpreting HER2 expression. Additionally, we identified a "grey-zone" among pathologists, highlighting the limitations of manual subjective interpretation. AI assistance within this zone may help mitigate subjective variations. Table 1. Correlation between pathologist read and AI read of HER2 immunohistochemistry. Table 2. Correlation between pathologist rescoring and AI read of HER2 immunohistochemistry. Citation Format: Ming Li, Hong Lv, Yizhi Zhao, Chenglu Zhu, Hansheng Li, Mingzhen Lin, Wen-Tao Yang. The Advantage of Artificial-Intelligence in HER2 IHC 0 and 1+ Scoring in Breast Cancer [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr PO4-26-09.

Forensic DNA Profiling: Legal and Ethical Considerations

Forensic DNA analysis is vital in criminal investigations, posing intricate legal and ethical dilemmas. This emphasises the need to balance DNA technology benefits with individual rights, prioritizing privacy, consent, and ethical practices. Stringent regulation of DNA data management is essential to prevent misuse and ensure privacy protection. Advances like probabilistic genotyping software improve interpretation accuracy, addressing complexities in mixture analysis. Responsible DNA use also requires navigating emerging technologies like next-generation sequencing while upholding ethical standards. Establishing transparency, accuracy, and ethical conduct throughout the analysis process is crucial for building public trust. Regulatory oversight and accountability mechanisms are necessary to address errors and maintain fairness in DNA data management. Through continuous dialogue and adherence to ethical standards, the forensic community can uphold scientific integrity and preserve public confidence in DNA analysis.

A REVIEW OF CHAN PLOT APPLICATION AND RECENT ADVANCED MODELS FOR DIAGNOSING EXCESSIVE WATER PRODUCTION

Water is one of the major fluids associated with the operational cycle of the oil industry that must be carefully considered due to its environmental, treatment facility, and economic impacts. Over the years, various methods have been developed to identify excessive water production. These methods range from reliable and expensive ones, such as well-logging records, to less accurate methods that utilize available production and water-oil ratio data, such as the Chan plot. The Chan plot emphasizes that well production can exhibit various patterns of excessive water production, including constant water-oil ratios, normal displacement, channeling, and coning. However, manual interpretation of these plots is often confusing due to the noise present in the actual data. Machine learning models have improved interpretation accuracy, but limitations remain in detecting evolving water production patterns. This paper reviews the application of Chan plots and their integration with existing diagnostic tools for diagnosing excessive water production. It then focuses on a recent advanced model that leverages machine learning specifically designed to improve the interpretation of Chan plots. The review highlights the limitations of traditional interpretation techniques and explores how the recent advanced model can address these limitations. Additionally, the paper briefly discusses the potential of an interactive model for the continuous monitoring of water production patterns. Finally, the paper offers recommendations for future research directions.

Abstract 14455: Impact of Computer-Interpreted ECGs on the Accuracy of Medical Professionals

Background: The impact of computer ECG interpretation (CEI) on medical professionals’ performance is understudied. This study aims to assess the interpretation proficiency of diverse medical professionals with and without access to CEI reports. Methods: Medical professionals from diverse backgrounds interpreted 60 12-lead ECGs with urgent and non-urgent findings. The interpretation process involved two interpretation phases: (i) 30 ECGs with clinical statements and (ii) the same 30 ECGs and clinical statements in random order along with a CEI report. Diagnostic performance was evaluated based on interpretation accuracy, time per ECG, and self-reported confidence (rated 0 [not confident], 1 [somewhat confident], or 2 [confident]). Results: Among 892 participants, there were 44 (4.9%) primary care physicians, 123 (13.8%) cardiology fellows-in-training, 259 (29.0%) resident physicians, 137 (15.4%) medical students, 56 (6.3%) advanced practice providers, 82 (9.2%) nurses, and 191 (21.4%) allied health professionals. The inclusion of the CEI significantly improved interpretation accuracy by 15.1% (95% CI, 14.3 to 16.0; P<0.001), reduced interpretation time by 52 seconds (95% CI, -56 to -48; P<0.001), and increased confidence by 0.06 (95% CI, 0.03 to 0.09; P=0.003) (Table 1) . Conclusions: CEI integration improves ECG interpretation accuracy, efficiency, and confidence across medical professionals. Nevertheless, even with CEI, significant gaps in ECG interpretation proficiency remain.

Differences and similarities in false interpretations by AI CAD and radiologists in screening mammography.

We aimed to evaluate the false interpretations between artificial intelligence (AI) and radiologists in screening mammography to get a better understanding of how the distribution of diagnostic mistakes might change when moving from entirely radiologist-driven to AI-integrated breast cancer screening. This retrospective case-control study was based on a mammography screening cohort from 2008 to 2015. The final study population included screening examinations for 714 women diagnosed with breast cancer and 8029 randomly selected healthy controls. Oversampling of controls was applied to attain a similar cancer proportion as in the source screening cohort. We examined how false-positive (FP) and false-negative (FN) assessments by AI, the first reader (RAD 1) and the second reader (RAD 2), were associated with age, density, tumor histology and cancer invasiveness in a single- and double-reader setting. For each reader, the FN assessments were distributed between low- and high-density females with 53 (42%) and 72 (58%) for AI; 59 (36%) and 104 (64%) for RAD 1 and 47 (36%) and 84 (64%) for RAD 2. The corresponding numbers for FP assessments were 1820 (47%) and 2016 (53%) for AI; 1568 (46%) and 1834 (54%) for RAD 1 and 1190 (43%) and 1610 (58%) for RAD 2. For ductal cancer, the FN assessments were 79 (77%) for AI CAD; with 120 (83%) for RAD 1 and with 96 (16%) for RAD 2. For the double-reading simulation, the FP assessments were distributed between younger and older females with 2828 (2.5%) and 1554 (1.4%) for RAD 1 + RAD 2; 3850 (3.4%) and 2940 (2.6%) for AI+RAD 1 and 3430 (3%) and 2772 (2.5%) for AI+RAD 2. The most pronounced decrease in FN assessments was noted for females over the age of 55 and for high density-women. In conclusion, AI could have an important complementary role when combined with radiologists to increase sensitivity for high-density and older females. Our results highlight the potential impact of integrating AI in breast cancer screening, particularly to improve interpretation accuracy. The use of AI could enhance screening outcomes for high-density and older females.

Transformer's frequency response analysis results interpretation using a novel cross entropy based methodology

Transformer defects can be identified by the FRA (frequency response analysis) that is a promising diagnostic technique. Despite the standardization in FRA measuring technique, its results interpretation is yet a research area. Because different faults types can be identified in various frequency bounds of the FRA signatures, it is necessary to identify the possible relationships between specific failures and frequency ranges in this contribution. For this purpose, a real transformer is used to conduct the essential tests, which include both healthy and faulted circumstances (axial displacement (AD), radial deformation (RD), and short-circuits (SC)). To identify efficient characteristics from the produced frequency response traces and improve interpretation accuracy of such traces, a new hyperbolic fuzzy cross entropy (FCE) measure is demonstrated and then utilized for the aim of discrimination and classification of transformer winding defects in pre-defined frequency ranges. After normalizing FRA results of the transformer under healthy and various fault circumstances the lower bounds from such responses have been extracted and then utilized to construct the desired form of the fuzzy sets of healthy and faulted circumstances. Then, a new hyperbolic FCE measure-based discrimination and classification of winding faults methodology is offered on the basis of highest and lowest FCE measure values. The highest FCE measure value between the fuzzy sets of healthy and faulted circumstances such as AD, RD and SC is designated to confirm the occurrence of winding faults in a suitable frequency range. The suggested methodology ensures smart interpretation of FRA signature and accurate classification of winding faults as it can effectively discriminate both healthy and faulted circumstances in the desired frequency ranges. The proposed approaches' performance is tested and compared by applying the experimental data after feature extraction.

Machine Learning Augmented Interpretation of Chest X-rays: A Systematic Review

Limitations of the chest X-ray (CXR) have resulted in attempts to create machine learning systems to assist clinicians and improve interpretation accuracy. An understanding of the capabilities and limitations of modern machine learning systems is necessary for clinicians as these tools begin to permeate practice. This systematic review aimed to provide an overview of machine learning applications designed to facilitate CXR interpretation. A systematic search strategy was executed to identify research into machine learning algorithms capable of detecting >2 radiographic findings on CXRs published between January 2020 and September 2022. Model details and study characteristics, including risk of bias and quality, were summarized. Initially, 2248 articles were retrieved, with 46 included in the final review. Published models demonstrated strong standalone performance and were typically as accurate, or more accurate, than radiologists or non-radiologist clinicians. Multiple studies demonstrated an improvement in the clinical finding classification performance of clinicians when models acted as a diagnostic assistance device. Device performance was compared with that of clinicians in 30% of studies, while effects on clinical perception and diagnosis were evaluated in 19%. Only one study was prospectively run. On average, 128,662 images were used to train and validate models. Most classified less than eight clinical findings, while the three most comprehensive models classified 54, 72, and 124 findings. This review suggests that machine learning devices designed to facilitate CXR interpretation perform strongly, improve the detection performance of clinicians, and improve the efficiency of radiology workflow. Several limitations were identified, and clinician involvement and expertise will be key to driving the safe implementation of quality CXR machine learning systems.

Inclusion of a Visual Aid Improves Surgeon and Trainee Accuracy when Evaluating Images for Ankle Syndesmosis Repair

Category:Trauma; Ankle;OtherIntroduction/Purpose:Malreduction in ankle syndesmosis repair is associated with post-operative pain, instability, and revision. Although multiple surgical techniques are designed to minimize malreduction, little is understood regarding the multi-factorial sources of error these techniques aim to address. For example, image interpretation errors may be particularly problematic, as previous work indicates identifying subtle fibular displacements from imaging is challenging for experienced orthopaedic trauma surgeons. In this context, this study evaluates whether surgical experience and/or inclusion of a visual aid improves surgeon accuracy during an image interpretation task. The task originates from the center-center technique, which is designed to minimize syndesmosis malreduction by drilling through the aligned centers of the tibia and fibula on an internally-rotated, lateral ankle x-ray.Methods:Twenty-eight individuals participated in this IRB-approved study (10 orthopaedic surgery attendings, 8 orthopaedic surgery residents, 10 medical students). Each evaluated 90 lateral ankle images arranged in two equal sets. In each set, the fibula was displaced from perfectly centered (-40% posterior to +40% anterior in 10% increments; each increment repeated 5 times). The two sets were identical, except one was augmented with a visual aid (or crosshair) marking the center of the tibia. Image and set order were randomized to control for learning effects. For each image, participants classified the fibula as 'centered' or 'non- centered' while wearing eye-tracking glasses. The glasses recorded two physiological measures of cognitive burden: blinks per image and image interpretation time. Statistical analyses tested the hypotheses that surgical experience and addition of a visual aid improved interpretation accuracy and decreased cognitive burden. Statistical significance (p <= 0.05) was evaluated using ANOVA followed by paired t-tests.Results:All participants accurately identified images with large fibular displacements (>20%) and perfectly centered (0%) images; however, interpretation accuracy at small displacements (10%) approximated random chance (Fig. 1A). Inclusion of a visual aid significantly improved image interpretation accuracy for both the full image set (p = 0.015, Fig. 1B) and images with small displacements (p = 0.025, Fig. 1A 10% and 20%). Although surgical experience was not a significant predictor of interpretation accuracy, surgical experience did significantly influence physiological measures of cognitive burden. Interestingly, inclusion of a visual aid slightly increased cognitive burden as measured by blinks per image for trainees, but significantly decreased this metric for attendings (p = 0.014, Fig. 1C). Compared to attendings, time per image was significantly increased for residents (p = 0.007), but not medical students (Fig. 1D).Conclusion:This study demonstrates that a visual aid can improve image interpretation accuracy for both experienced surgeons and trainees. Interestingly, the inclusion of the visual aid decreased the physiologically measured cognitive burden of attendings, but not trainees. This suggests that additional visual information displayed on an x-ray image may be processed and used differently by experienced versus inexperienced surgeons. Thus, this study provides foundational evidence that augmenting images can improve surgical accuracy and motivates future work to characterize how image augmentation influences clinical and educational outcomes.

The effect of clinical information on radiology reporting: A systematic review.

IntroductionThe aim of this study was to investigate the effects of clinical information on the accuracy, timeliness, reporting confidence and clinical relevance of the radiology report.MethodsA systematic review of studies that investigated a link between primary communication of clinical information to the radiologist and the resultant report was conducted. Relevant studies were identified by a comprehensive search of electronic databases (PubMed, Scopus and EMBASE). Studies were screened using pre‐defined criteria. Methodological quality was assessed using the Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Quasi‐Experimental Studies. Synthesis of findings was narrative. Results were reported according to the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) guidelines.ResultsThere were 21 studies which met the inclusion criteria, of which 20 were included in our review following quality assessment. Sixteen studies investigated the effect of clinical information on reporting accuracy, three studies investigated the effect of clinical information on reporting confidence, three studies explored the impact of clinical information on clinical relevance, and two studies investigated the impact of clinical information on reporting timeliness. Some studies explored multiple outcomes. Studies concluded that clinical information improved interpretation accuracy, clinical relevance and reporting confidence; however, reporting time was not substantially affected by the addition of clinical information.ConclusionThe findings of this review suggest clinical information has a positive impact on the radiology report. It is in the best interests of radiologists to communicate the importance of clinical information to reporting via the creation of criteria standards to guide the requesting practices of medical imaging referrers. Further work is recommended to establish these criteria standards.

Imaging evaluation of diaphragmatic injuries: Improving interpretation accuracy

Diaphragmatic Injuries (DIs) remain a challenging diagnosis with potential catastrophic delayed complications. A high degree of suspicion in every case of severe blunt thoracoabdominal trauma or penetrating thoracoabdominal injury is essential. This review will present the evidence and controversies on this topic providing a practical tutorial for radiologists hoping to improve their interpretive accuracy for both blunt and penetrating DIs. The imaging signs of diaphragmatic injuries will be explained with emphasis on multidetector CT. Diagnostic pitfalls, available protocols and other issues will be presented.

Regularization strategies for deep-learning-based salt model building

Salt model building has long been considered a severe bottleneck for large-scale 3D seismic imaging projects. It is one of the most time-consuming, labor-intensive, and difficult-to-automate processes in the entire depth imaging workflow requiring significant intervention by domain experts to manually interpret the salt bodies on noisy, low-frequency, and low-resolution seismic images at each iteration of the salt model building process. The difficulty and need for automating this task is well-recognized by the imaging community and has propelled the use of deep-learning-based convolutional neural network (CNN) architectures to carry out this task. However, significant challenges remain for reliable production-scale deployment of CNN-based methods for salt model building. This is mainly due to the poor generalization capabilities of these networks. When used on new surveys, never seen by the CNN models during the training stage, the interpretation accuracy of these models drops significantly. To remediate this key problem, we have introduced a U-shaped encoder-decoder type CNN architecture trained using a specialized regularization strategy aimed at reducing the generalization error of the network. Our regularization scheme perturbs the ground truth labels in the training set. Two different perturbations are discussed: one that randomly changes the labels of the training set, flipping salt labels to sediments and vice versa and the second that smooths the labels. We have determined that such perturbations act as a strong regularizer preventing the network from making highly confident predictions on the training set and thus reducing overfitting. An ensemble strategy is also used for test time augmentation that is shown to further improve the accuracy. The robustness of our CNN models, in terms of reduced generalization error and improved interpretation accuracy is demonstrated with real data examples from the Gulf of Mexico.

Application of Borehole Radar Data Processing Based on Empirical Mode Decomposition

Borehole Radar (BHR) uses ultra-wideband electromagnetic (EM) waves to image discontinuities in formations. It has been a major bottleneck to extend BHR applications to obtain a clear and high-resolution radar profile in a complex and noisy environment, which increases ambiguity in the geology interpretation. To avoid this increased ambiguity in the geology interpretation, we proposed a scheme based on the empirical mode decomposition (EMD) and complex signal analysis theory to process the BHR data with low signal to noise ratio (SNR). The scheme includes four steps. First, the original radar profile is pre-processed to avoid mode confusion and noise interference to the radar echo. Next, the EMD method is used to process a single-channel radar dataset and to analyze the frequency components of the radar signal. Various intrinsic modes of the pre-processing radar profile are also obtained by using EMD. Finally, we reconstruct the intrinsic mode profile, which contains information about the formation, calculate the complex signals of the reconstructed radar profile using the Hilbert transform, extract the three instantaneous attributes (instantaneous amplitude, instantaneous phase, and instantaneous frequency), and draw the separate instantaneous attributes profiles. This processing scheme provides both the conventional time-distance profile also in addition to the three instantaneous attributes. The additional attributes reduce ambiguity when evaluating the original radar profile and avoid the deviation relying solely on a conventional time-distance profile. An actual radar profile, which was obtained by a BHR system in a limestone fracture zone, is used to verify the effectiveness of instantaneous attributes for improving interpretation accuracy. The results demonstrate that the EMD method is superior in processing the BHR signal under a low SNR and has the capability to separate the high-low components of the radar echo effectively.

A decision support system and rule-based algorithm to augment the human interpretation of the 12-lead electrocardiogram

BackgroundThe 12-lead Electrocardiogram (ECG) has been used to detect cardiac abnormalities in the same format for more than 70years. However, due to the complex nature of 12-lead ECG interpretation, there is a significant cognitive workload required from the interpreter. This complexity in ECG interpretation often leads to errors in diagnosis and subsequent treatment. We have previously reported on the development of an ECG interpretation support system designed to augment the human interpretation process. This computerised decision support system has been named ‘Interactive Progressive based Interpretation’ (IPI). In this study, a decision support algorithm was built into the IPI system to suggest potential diagnoses based on the interpreter's annotations of the 12-lead ECG. We hypothesise semi-automatic interpretation using a digital assistant can be an optimal man-machine model for ECG interpretation. ObjectivesTo improve interpretation accuracy and reduce missed co-abnormalities. MethodsThe Differential Diagnoses Algorithm (DDA) was developed using web technologies where diagnostic ECG criteria are defined in an open storage format, Javascript Object Notation (JSON), which is queried using a rule-based reasoning algorithm to suggest diagnoses. To test our hypothesis, a counterbalanced trial was designed where subjects interpreted ECGs using the conventional approach and using the IPI+DDA approach. ResultsA total of 375 interpretations were collected. The IPI+DDA approach was shown to improve diagnostic accuracy by 8.7% (although not statistically significant, p-value=0.1852), the IPI+DDA suggested the correct interpretation more often than the human interpreter in 7/10 cases (varying statistical significance). Human interpretation accuracy increased to 70% when seven suggestions were generated. ConclusionAlthough results were not found to be statistically significant, we found; 1) our decision support tool increased the number of correct interpretations, 2) the DDA algorithm suggested the correct interpretation more often than humans, and 3) as many as 7 computerised diagnostic suggestions augmented human decision making in ECG interpretation. Statistical significance may be achieved by expanding sample size.

Effect of distinguishing apparent permeability on flowing gas composition, composition change and composition derivative in tight- and shale-gas reservoir

Multi-flow mechanisms co-exist in tight- and shale-gas reservoirs such as slip flow, transition flow, and free molecular flow, thus apparent permeability is used to correct the flow deviation caused by the traditional Darcy’s law. Strictly speaking, apparent permeability is also different for different gas components, which we call as multi-component permeability (MCP). In this paper, we use a compositional model incorporating extended Langmuir isotherm and slippage corrected MCP to study the effect of distinguishing apparent permeability for different components on the curves of wellhead flowing composition (WHFC) and its derivative. A fully implicit iterative numerical solution based on PEBI (Perpendicular Bisection) gridding generates 12 simulations to investigate WHFC, composition change (dc) and composition derivative (dc′) with and without using MCP for a fractured vertical well in the reservoirs with different adsorption capacities. We find that MCP is an important factor for CH4 component variance, especially CH4 component variance is solely caused by MCP for reservoirs without adsorption gas, which shows that MCP can improve interpretation accuracy based on the measured composition. Another finding is that the behavior of WHFC is obviously different between reservoirs with and without adsorbed gas, which can be used to determine whether reservoirs contain adsorption gas or not.

Edge enhancement of gravity anomalies and gravity gradient tensors using an improved small sub-domain filtering method

In order to enhance geological body boundary visual effects in images and improve interpretation accuracy using gravity and magnetic field data, we propose an improved small sub-domain filtering method to enhance gravity anomalies and gravity gradient tensors. We discuss the effect of Gaussian white noise on the improved small sub-domain filtering method, as well as analyze the effect of window size on geological body edge recognition at different extension directions. Model experiments show that the improved small sub-domain filtering method is less affected by noise, filter window size, and geological body edge direction so it can more accurately depict geological body edges than the conventional small sub-domain filtering method. It also shows that deeply buried body edges can be well delineated through increasing the filter window size. In application, the enhanced gravity anomalies and calculated gravity gradient tensors of the Hulin basin show that the improved small sub-domain filtering can recognize more horizontal fault locations than the conventional method.

Pictographic instructions for medications: do different cultures interpret them accurately?

Abstract Objective (1) To determine whether a range of linguistically diverse individuals of non-European descent could understand the meaning of predeveloped pictograms without any additional aids such as verbal explanations; (2) to identify appropriate modifications to the pictograms to diminish errors of interpretation; and (3) to test the notion that central pictogram elements have the same meaning irrespective of language or culture. Setting The study was conducted in Vancouver, Canada. Method This study tested a set of 16 predeveloped pictograms to determine whether they accurately communicated the intended medication administration directions to participants who neither speak nor read English, French or Spanish. Both qualitative and quantitative methods evaluated the pictograms' interpretability among 39 participants from three language groups, Cantonese, Somali and Punjabi. Standard analysis of variance tested for differences due to language groups and other demographics. Key findings Only four of the 16 initial pictograms tested were interpreted correctly by 70% of participants. Relaxing the criterion from 70% to 50% included seven more. Modifications of problem elements within the pictograms further improved interpretation accuracy levels by 22%, to a final accuracy of 67.15%. Quantity errors were twice as common as timing, administration route or auxiliary instruction errors. Conclusions Participants could identify particular pictographic symbols they found confusing or ambiguous. Basic education and time since immigration predicted interpretation accuracy better than first language or any other demographic characteristic.