Human-AI perception: not much different, but some distinct novelties

We aimed to investigate current expectations and clinical adoption of artificial intelligence (AI) software among neuroradiologists in Korea. In April 2022, a 30-item online survey was conducted by neuroradiologists from the Korean Society of Neuroradiology (KSNR) to assess current user experiences, perceptions, attitudes, and future expectations regarding AI for neuro-applications. Respondents with experience in AI software were further investigated in terms of the number and type of software used, period of use, clinical usefulness, and future scope. Results were compared between respondents with and without experience with AI software through multivariable logistic regression and mediation analyses. The survey was completed by 73 respondents, accounting for 21.9% (73/334) of the KSNR members; 72.6% (53/73) were familiar with AI and 58.9% (43/73) had used AI software, with approximately 86% (37/43) using 1-3 AI software programs and 51.2% (22/43) having up to one year of experience with AI software. Among AI software types, brain volumetry software was the most common (62.8% [27/43]). Although 52.1% (38/73) assumed that AI is currently useful in practice, 86.3% (63/73) expected it to be useful for clinical practice within 10 years. The main expected benefits were reducing the time spent on repetitive tasks (91.8% [67/73]) and improving reading accuracy and reducing errors (72.6% [53/73]). Those who experienced AI software were more familiar with AI (adjusted odds ratio, 7.1 [95% confidence interval, 1.81-27.81]; P = 0.005). More than half of the respondents with AI software experience (55.8% [24/43]) agreed that AI should be included in training curriculums, while almost all (95.3% [41/43]) believed that radiologists should coordinate to improve its performance. A majority of respondents experienced AI software and showed a proactive attitude toward adopting AI in clinical practice, suggesting that AI should be incorporated into training and active participation in AI development should be encouraged.

To evaluate the performance of a commercial artificial intelligence (AI) software in detecting intracranial hemorrhage (ICH) in emergency settings, compared to on-call radiology residents. All consecutive unenhanced cerebral CT-scans performed in a single center over a 3-month period in the emergency department in patients with suspected ICH, initially interpreted by radiology residents on-call and subsequently verified and approved by a board-certified radiologist, were concomitantly analyzed by an AI software for the presence of ICH. Results from the AI software were stored in a separate PACS partition and were unavailable to the radiologists for the case reading. We assessed the diagnostic performance of the AI software and of the radiology residents in detecting ICH. The reference standard was the final report of the board-certified radiologist. Radiology reports of 2153 CT-scans were analyzed, and ICH prevalence was 15.4% (331/2153). The AI software achieved an overall sensitivity of 84% and a specificity of 94.4%, and radiology residents achieved a sensitivity of 96.4% and a specificity of 99.6%, respectively (p-values < 0.001). The sensitivity was 97.7% for AI and 98.5% for residents when CT examinations displayed an association of multiple hemorrhagic types (p = 1). The sensitivity was 95.2% for AI and 98.4% for radiology residents in the presence of multiple ICH sites (p = 0.11). Radiology residents demonstrated a significantly higher performance in detecting ICH compared to the AI software. AI exhibited very good diagnostic performance only in the presence of multiple hemorrhagic sites or multiple hemorrhage types. QuestionHow does the performance of the AI software compare to that of radiology residents in detecting ICH on unenhanced CT in real-life emergency workflow conditions? FindingsIn the emergency setting, the AI software demonstrated lower overall sensitivity and specificity than radiology residents for detecting ICH. Clinical relevanceIn real-life emergency conditions at a university hospital, the AI software did not offer a superior performance compared to radiology residents in detecting ICH. The integration of AI in this specific setting remains to be defined.

BackgroundTo demonstrate the value of an artificial intelligence (AI) software in the detection of mammographically occult breast cancers and to determine the clinicopathologic patterns of the cancers additionally detected using the AI software.MethodsBy retrospectively reviewing our institutional database (January 2017–September 2019), we identified women with mammographically occult breast cancers and analyzed their mammography with an AI software that provided a malignancy score (range 0–100; > 10 considered as positive). The hot spots in the AI report were compared with the US and MRI findings to determine if the cancers were correctly marked by the AI software. The clinicopathologic characteristics of the AI-detected cancers were analyzed and compared with those of undetected cancers.ResultsAmong the 1890 breast cancers, 6.8% (128/1890) were mammographically occult, among which 38.3% (49/128) had positive results in the AI analysis. Of them, 81.6% (40/49) were correctly marked by the AI software and determined as “AI-detected cancers.” As such, 31.3% (40/128) of mammographically occult breast cancers could be identified by the AI software. Of the AI-detected cancers, 97.5% were found in heterogeneously or extremely dense breasts, 52.5% were asymptomatic, 86.5% were invasive, and 29.7% had axillary lymph node metastasis. Compared with undetected cancers, the AI-detected cancers were more likely to be found in younger patients (p < 0.001), undergo neoadjuvant chemotherapy as well as mastectomy rather than breast-conserving operation (both p < 0.001), and accompany axillary lymph node metastasis (p = 0.003).ConclusionsAI conferred an added value in the detection of mammographically occult breast cancers.

Artificial intelligence (AI) promises to change health care, with some studies showing proof of concept of a provider-level performance in various medical specialties. However, there are many barriers to implementing AI, including patient acceptance and understanding of AI. Patients' attitudes toward AI are not well understood. We systematically reviewed the literature on patient and general public attitudes toward clinical AI (either hypothetical or realised), including quantitative, qualitative, and mixed methods original research articles. We searched biomedical and computational databases from Jan 1, 2000, to Sept 28, 2020, and screened 2590 articles, 23 of which met our inclusion criteria. Studies were heterogeneous regarding the study population, study design, and the field and type of AI under study. Six (26%) studies assessed currently available or soon-to-be available AI tools, whereas 17 (74%) assessed hypothetical or broadly defined AI. The quality of the methods of these studies was mixed, with a frequent issue of selection bias. Overall, patients and the general public conveyed positive attitudes toward AI but had many reservations and preferred human supervision. We summarise our findings in six themes: AI concept, AI acceptability, AI relationship with humans, AI development and implementation, AI strengths and benefits, and AI weaknesses and risks. We suggest guidance for future studies, with the goal of supporting the safe, equitable, and patient-centred implementation of clinical AI.

Assessment of Radiology Artificial Intelligence Software: A Validation and Evaluation Framework.

Use of Artificial Intelligence Software to Detect Intracranial Aneurysms: A Comprehensive Stroke Center Experience

IntroductionAs human medical diagnostic expertise is scarcely available, especially in veterinary care, artificial intelligence (AI) has been increasingly used as a remedy. AI's promise comes from improving human diagnostics or providing good diagnostics at lower cost, increasing access. This study analyzed the diagnostic performance of a widely used AI radiology software vs. veterinary radiologists in interpreting canine and feline radiographs. We aimed to establish whether the performance of commonly used AI matches the performance of a typical radiologist and thus can be reliably used. Secondly, we try to identify in which cases AI is effective.MethodsFifty canine and feline radiographic studies in DICOM format were anonymized and reported by 11 board-certified veterinary radiologists (ECVDI or ACVR) and processed with commercial and widely used AI software dedicated to small animal radiography (SignalRAY®, SignalPET® Dallas, TX, USA). The AI software used a deep-learning algorithm and returned a coded abnormal or normal diagnosis for each finding in the study. The radiologists provided a written report in English. All reports' findings were coded into categories matching the codes from the AI software and classified as normal or abnormal. The sensitivity, specificity, and accuracy of each radiologist and the AI software were calculated. The variance in agreement between each radiologist and the AI software was measured to calculate the ambiguity of each radiological finding.ResultsAI matched the best radiologist in accuracy and was more specific but less sensitive than human radiologists. AI did better than the median radiologist overall in low- and high-ambiguity cases. In high-ambiguity cases, AI's accuracy remained high, though it was less effective at detecting abnormalities but better at identifying normal findings. The study confirmed AI's reliability, especially in low-ambiguity scenarios.ConclusionOur findings suggest that AI performs almost as well as the best veterinary radiologist in all settings of descriptive radiographic findings. However, its strengths lie more in confirming normality than detecting abnormalities, and it does not provide differential diagnoses. Therefore, the broader use of AI could reliably increase diagnostic availability but requires further human input. Given the unique strengths of human experts and AI and the differences in sensitivity vs. specificity and low-ambiguity vs. high-ambiguity settings, AI will likely complement rather than replace human experts.

This study investigates the adoption of artificial intelligence (AI) software in manufacturing companies in Slovenia, Slovakia and Croatia, and across six production areas. This research ad-dresses a gap in the literature regarding AI software implementation in relation to company size, technology intensity and supply chain role, and examines whether Industry 4.0 (I4.0) readiness influences AI adoption. Data from the European Manufacturing Survey 2022 were analyzed, and showed that the use of AI is still relatively low. On average only 18.4% of companies use AI software in at least one production area. Logistic regression analysis revealed that neither company size nor role in the supply chain or technology intensity are statistically significantly related to AI usage. However, a significant positive relationship was found between I4.0 readiness and AI adoption, suggesting that companies with advanced digital infrastructures and integrated cyber-physical systems are more likely to adopt AI. This finding underlines the importance of digital transformation for the integration of AI software. The study concludes that while company characteristics such as size and the role of the company in the supply chain are not statistically significantly related to the use of AI, the level of digital readiness is crucial.

Objective: Ki-67 Label Index (Ki-67LI) is a breast cancer(BC) predictive and prognostic factor. The lack of standardization and reproducibility of evaluation methods limits its use in routine work. In this study, Ki-67 standard comparison card (SRC) and artificial intelligence(AI) software were used to evaluate breast cancer Ki-67LI. We established training and validation sets and studied the repeatability between observers. Methods: A total of 300 invasive breast cancer specimens were randomly divided into training and verification sets, with each set including 150 cases. Breast cancer Ki-67 standard comparison cards ranging from 5% to 90% were created. The training set was interpreted by nine pathologists of different ages through microscopic visual assessment (VA), SRC, microscopic manual counting (MC), and AI. The validation set was interpreted by three randomly selected pathologists using SRC and AI. Friedman M was used to analyze the difference. The intra-group correlation coefficient (ICC) and Bland-Altman scatter plot were used for consistency analysis. Results: 1.Ki-67LI interpreted by the four methods in the training set did not obey a normal distribution (P&amp;lt;0.05). Friedman M test showed that the difference between pathologists using the same method was statistically significant (P&amp;lt;0.05). After Bonferroni correction, Ki-67LI interpreted using SRC and AI showed that the difference between each pathologist and the gold standard was statistically significant (P&amp;lt;0.05), and the difference between pathologists was not statistically significant (P&amp;gt;0.05); Ki-67LI interpreted using VA and MC showed that the difference between each pathologist and the gold standard and the difference between pathologists were statistically significant (P&amp;lt;0.05). 2. The intra-group correlation coefficient(ICC) obtained by nine pathologists in the training set that used SRC (ICC=0.918) and AI (ICC=0.972) to interpret Ki-67LI, was significantly higher than when VA (ICC=0.757) and MC (ICC=0.803) were used. 3. Through SRC, the initial and intermediate pathologists in the training set had an increased ICC. 4. In the homogeneous group of the training set, the agreement on observers of VA, MC, SRC, and AI among observes was very good, with all ICC values above 0.80. In the heterogeneous group, SRC and AI showed a good agreement among observers (ICC= 0.877 and 0.959, respectively). In the homogeneous and heterogeneous groups of validation sets, the consistency among the pathologists that used SRC and AI was very good, with an ICC of&amp;gt;0.90. 5. In the verification set, using SRC and AI, three pathologists obtained results that were very consistent with the gold standard, having an ICC above 0.95, and the inter-observer agreement was also very good, with an ICC of&amp;gt;0.9. Conclusion: AI has satisfactory inter-observer repeatability, and the true value was closer to the gold standard, which is the preferred method for Ki-67LI reproducibility; While AI software has not been popularized, SRC may be interpreted as breast cancer Ki-67LI's standard candidate method.Keywords: Breast cancer, Ki-67, Artificial intelligence, Ki-67 standard comparison card, Repeatability Citation Format: Lina Li, Yueping Liu. Artificial intelligence-assisted interpretation of Ki-67 expression and repeatability in breast cancer [abstract]. In: Proceedings of the 2020 San Antonio Breast Cancer Virtual Symposium; 2020 Dec 8-11; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2021;81(4 Suppl):Abstract nr PS2-29.

Angular and longitudinal deformities of leg alignment create excessive stresses across joints, leading to pain and impaired function. Multiple measurements are used to assess these deformities on anteroposterior (AP) full-length radiographs. An artificial intelligence (AI) software automatically locates anatomical landmarks on AP full-length radiographs and performs 13 measurements to assess knee angular alignment and leg length. The primary aim of this study was to evaluate the agreements in LLD and knee alignment measurements between an AI software and two board-certified radiologists in patients without metal implants. The secondary aim was to assess time savings achieved by AI. The measurements assessed in the study were hip-knee-angle (HKA), anatomical-tibiofemoral angle (aTFA), anatomical-mechanical-axis angle (AMA), joint-line-convergence angle (JLCA), mechanical-lateral-proximal-femur-angle (mLPFA), mechanical-lateral-distal-femur-angle (mLDFA), mechanical-medial-proximal-tibia-angle (mMPTA), mechanical-lateral-distal-tibia- angle (mLDTA), femur length, tibia length, full leg length, leg length discrepancy (LLD), and mechanical axis deviation (MAD). These measurements were performed by two radiologists and the AI software on 164 legs. Intraclass-correlation-coefficients (ICC) and Bland-Altman analyses were used to assess the AI's performance. The AI software set incorrect landmarks for 11/164 legs. Excluding these cases, ICCs between the software and radiologists were excellent for 12/13 variables (11/13 with outliers included), and the AI software met performance targets for 11/13 variables (9/13 with outliers included). The mean reading time for the AI algorithm and two readers, respectively, was 38.3, 435.0, and 625.0s. This study demonstrated that, with few exceptions, this AI-based software reliably generated measurements for most variables in the study and provided substantial time savings.

Despite recent advances in artificial intelligence (AI) software with improved performance in mammography screening for breast cancer, insufficient data are available on its performance in detecting cancers that were initially missed on mammography. In this study, we aimed to determine whether AI software-aided mammography could provide additional value in identifying cancers detected through supplemental screening ultrasound. We searched our database from 2017 to 2018 and included 238 asymptomatic patients (median age, 50 years; interquartile range, 45–57 years) diagnosed with breast cancer using supplemental ultrasound. Two unblinded radiologists retrospectively reviewed the mammograms using commercially available AI software and identified the reasons for missed detection. Clinicopathological characteristics of AI-detected and AI-undetected cancers were compared using univariate and multivariate logistic regression analyses. A total of 253 cancers were detected in 238 patients using ultrasound. In an unblinded review, the AI software failed to detect 187 of the 253 (73.9%) mammography cases with negative findings in retrospective observations. The AI software detected 66 cancers (26.1%), of which 42 (63.6%) exhibited indiscernible findings obscured by overlapping dense breast tissues, even with the knowledge of magnetic resonance imaging and post-wire localization mammography. The remaining 24 cases (36.4%) were considered interpretive errors by the radiologists. Invasive tumor size was associated with AI detection after multivariable analysis (odds ratio, 2.2; 95% confidence intervals, 1.5–3.3; p < 0.001). In the control group of 160 women without cancer, the AI software identified 19 false positives (11.9%, 19/160). Although most ultrasound-detected cancers were not detected on mammography with the use of AI, the software proved valuable in identifying breast cancers with indiscernible abnormalities or those that clinicians may have overlooked.

Making waves: The potential of generative AI in water utility operations.

This study aimed to develop and implement an artificial intelligence (AI) software to measure students' attention levels in real-time during class. Additionally, the research explored instructors' perspectives on the use of AI for monitoring attention in the classroom. The quantitative phase of the research was conducted during the fall semester of 2021-2022 with 396 students from the Forensic Informatics Department at Fırat University. The qualitative phase involved five instructors participating in the software application. In the quantitative phase, data were collected using the AI software, an attention control scale, and an attention labeling form completed by three experts. A semi-structured interview form was used for the qualitative phase. The AI algorithm was developed by utilizing publicly available online photos to classify students' attention levels. Results showed that the AI software achieved a 97.2% accuracy rate in classifying attention levels. A pilot study conducted with 10 students provided strong evidence that the AI software could accurately measure attention levels. In the final phase involving 396 students, the AI software demonstrated its capability to visually report students' real-time attention levels by tracking head positions. The findings indicate that AI software can effectively monitor and report students' attention levels to instructors during class, providing valuable feedback in real time.

In physical health examinations, breast sonography is a commonly used imaging method, but it can lead to repeated exams and unnecessary biopsy due to discrepancies among radiologists and health centers. This study explores the role of off-the-shelf artificial intelligence (AI) software in assisting radiologists to classify incidentally found breast masses in two health centers. Female patients undergoing breast ultrasound examinations with incidentally discovered breast masses were categorized according to the 5th edition of the Breast Imaging Reporting and Data System (BI-RADS), with categories 3 to 5 included in this study. The examinations were conducted at two municipal health centers from May 2021 to May 2023.The final pathological results from surgical resection or biopsy served as the gold standard for comparison. Ultrasonographic images were obtained in longitudinal and transverse sections, and two junior radiologists and one senior radiologist independently assessed the images without knowing the pathological findings. The BI-RADS classification was adjusted following AI assistance, and diagnostic performance was compared using receiver operating characteristic curves. A total of 196 patients with 202 breast masses were included in the study, with pathological results confirming 107 benign and 95 malignant masses. The receiver operating characteristic curve showed that experienced breast radiologists had higher diagnostic performance in BI-RADS classification than junior radiologists, similar to AI classification (AUC = 0.936, 0.806, 0.896, and 0.950, p < 0.05). The AI software improved the accuracy, sensitivity, and negative predictive value of the adjusted BI-RADS classification for the junior radiologists' group (p< 0.05), while no difference was observed in the senior radiologist group. Furthermore, AI increased the negative predictive value for BI-RADS 4a masses and the positive predictive value for 4b masses among radiologists (p < 0.05). AI enhances the sensitivity of invasive breast cancer detection more effectively than ductal carcinoma in situ and rare subtypes of breast cancer. The AI software enhances diagnostic efficiency for breast masses, reducing the performance gap between junior and senior radiologists, particularly for BI-RADS 4a and 4b masses. This improvement reduces unnecessary repeat examinations and biopsies, optimizing medical resource utilization and enhancing overall diagnostic effectiveness.

This study evaluates the effectiveness of artificial intelligence (AI) in mammography in a diverse population from a middle-income nation and compares it to traditional methods. A retrospective study was conducted on 543 mammograms of 467 Malays, 48 Chinese, and 28 Indians in a middle-income nation. Three breast radiologists interpreted the examinations independently in two reading sessions (with and without AI support). Breast density and BI-RADS categories were assessed, comparing the accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) results. Of 543 mammograms, 69.2% had lesions detected. Biopsies were performed on 25%(n=136), with 66(48.5%) benign and 70(51.5%) malignant. Substantial agreement in density assessment between the radiologist and AI software (κ =0.606, p < 0.001) and the BI-RADS category with and without AI (κ =0.74, p < 0.001). The performance of the AI software was comparable to the traditional methods. The sensitivity, specificity, PPV, and NPV or radiologists alone, radiologist + AI, and AI alone were 81.9%,90.4%,56.0%, and 97.1%; 81.0%, 93.1%,55.5%, and 97.0%; and 90.0%,76.5%,36.2%, and 98.1%, respectively. AI software enhances the accuracy of lesion diagnosis and reduces unnecessary biopsies, particularly for BI-RADS 4 lesions. The AI software results for synthetic were almost similar to the original 2D mammography, with AUC of 0.925 and 0.871, respectively. AI software may assist in the accurate diagnosis of breast lesions, enhancing the efficiency of breast lesion diagnosis in a mixed population of opportunistic screening and diagnostic patients. • The use of artificial intelligence (AI) in mammography for population-based breast cancer screening has been validated in high-income nations, with reported improved diagnostic performance. Our study evaluated the usage of an AI tool in an opportunistic screening setting in a multi-ethnic and middle-income nation. • The application of AI in mammography enhances diagnostic accuracy, potentially leading to reduced unnecessary biopsies. • AI integration into the workflow did not disrupt the performance of trained breast radiologists, as there is a substantial inter-reader agreement for BI-RADS category assessment and breast density.