Convolutional Neuronal Networks Research Articles

Abstract 169: Prevalence, immune checkpoint expression and spatial interplay of intraepithelial lymphocytes, macrophages, and dendritic cells in 4740 human carcinomas

Abstract Background: Although there is raising evidence that the density and the level of immune checkpoint expression of immune cell subpopulations that are in direct contact to the tumor cells (intraepithelial) can predict response to immune checkpoint therapy and patient’s outcome, a comprehensive assessment of intraepithelial immune cell subpopulations and their spatial interplay is lacking. Material and methods: To comprehensively assess the density, level of immune checkpoint expression, and spatial interplay of 48 immune checkpoint expressing intraepithelial leukocyte subpopulations in 46 carcinoma entities, 4740 tumor samples in a tissue microarray format were stained with 21 antibodies using our BLEACH&STAIN mfIHC approach. A deep learning-based framework comprising 18 different convolutional neuronal networks (U-Net and DeepLabv3+) were used for image analysis. Results: The mean intraepithelial immune cell density of CD8+ cytotoxic T-cells, CD4+ T-helper cells, FOXP3+ Tregs, CD20+ B-cells, M1/M2 macrophages and CD11c+ dendritic cells varied markedly between tumor entities and individual tumors. For example, 88 (±90) cells/mm2 were found in tubular breast cancer, 661 (±729) cells/mm2 in colorectal cancer, and 1280 (±1334) to 2325 (±2131) cells/mm2 in squamous cell cancers from various origins. The abundance of T-cell subpopulations, M2 macrophages, and dendritic cells was highly concordant (range r=0.49 to 0.92) across all tumor entities, while the density of M1 macrophages was unrelated (r=0.1, p<0.001). Unsupervised hierarchical clustering across all tumor entities revealed a cluster A of 634 patients from almost all different tumor entities with an exceptionally high density of intraepithelial immune cells, thus a highly inflamed tumor microenvironment that was characterized by the strongest interaction between intraepithelial CD8+ T-cell and CD4+ T-helper cells, M2 macrophages, as well as dendritic cells (DC, p<0.002 each) along with the highest expression level of TIM3, PD-1, CLTA-4 and PD-L1. In contrast, a cluster C of 2245 patients with the lowest intraepithelial immune cell infiltration was characterized by a significantly increased M1 proportion (p<0.001) and an enriched proportion of regulatory T-cells (p<0.001). Across all analyzed tumor entities, the intraepithelial highly inflamed cluster A was significantly linked to low pT (p<0.001). Conclusion: The data from this study provide a comprehensive characterization of intraepithelial immune cells across 46 different human carcinomas and identify an inflamed phenotype defined by strong interactions of intraepithelial CD8+ cytotoxic T-cells, CD4+ T-cells, dendritic cells, and M2 macrophages along with highest levels of TIM3, PD-1, PD-L1, and CTLA-4 expression that is linked to a favorable tumor phenotype. Citation Format: Zhihao Huang, Tim Mandelkow, Jan H. Müller, Andreas H. Marx, Till Krech, Guido Sauter, Julia Ebner, Maximilian Lennartz, Ronald Simon, Martina Kluth, Claudia Hube-Magg, Sarah Minner, Elena Bady, Niclas C. Blessin. Prevalence, immune checkpoint expression and spatial interplay of intraepithelial lymphocytes, macrophages, and dendritic cells in 4740 human carcinomas [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 169.

Abstract 1517: Tc1 cytotoxic T-cells and Th1 helper T-cell rich tumor microenvironment is a hallmark of MSI colorectal cancers

Abstract Background: Microsatellite instability (MSI-h) is a strong biomarker to predict response to immune checkpoint therapy and patient’s outcome in colorectal cancer. Although enrichment of distinct T-cell subpopulations, such as type 1 cytotoxic T-cells (Tc1) or type 1 helper T-cells (Th1), have been identified to impact patient’s outcome and response to immune checkpoint therapy, only little is known about the underlying changes in the composition of the immune tumor microenvironment. Material and Methods: To assess the density, composition, degree of functional marker expression, and spatial interplay of T-cell subpopulations in 93 MSI and 1153 microsatellite stable (MSS) colorectal cancers, a tissue microarray as well as large sections were stained with antibodies directed against CD3, CD4, CD8, FOXP3, T-bet, GATA3, RORyT, BCL6, CD27, CD56, TIM3, PD-1, CTLA-4 Granzym B, and Ki67 using our BLEACH&STAIN multiplex fluorescence immunohistochemistry approach. A deep learning-based framework comprising two different convolutional neuronal networks (U-Net and DeepLabv3+) was used for image analysis. Results: The composition of Type 1 (T-bet+), Type 2 (GATA3+), Type 17 (RORyT+), NKT-like (CD56+), regulatory (FOXP3+), and follicular (BCL6+) cytotoxic (CD3+CD8+) or helper (CD3+CD4+) T-cells showed marked differences between MSI and MSS patients. For instance, the density and proportion of Tc1 as well as Th1 was significantly higher (p<0.001 each) in MSI compared to MSS patients. In contrast, the density and composition of Tregs, Th2, Th17 T-cells was significantly lower in MSI compared to MSS patients in the intraepithelial tumor compartment. In addition, among the MSI colorectal cancers, NKT-cells showed the highest level of Granzyme B that was significantly higher than in MSS patients (p<0.05). The degree of immune checkpoint expression (i.e., TIM3, PD-1, CTLA-4) was significantly higher in MSI compared to MSS patients for most T-cell subpopulations (p<0.05 each). The additional analysis of 10 large sections confirmed the observations from the TMA analysis and revealed that the alterations in the T-cell composition seen in the center of the tumor largely reflect the T-cell composition at the invasive margin in both MSI and MSS patients. Conclusion: This study identified a higher proportion of Tc1 cytotoxic T-cells and Th1 helper T-cells accompanied by a paucity of regulatory T-cell subpopulations, Th17 and Th2 T- helper cells in MSI colorectal cancers compared to MSS patients. Citation Format: Tim Mandelkow, Zhihao Huang, Elena Bady, Jan H. Müller, Guido Sauter, Julia Ebner, Maximilian Lennartz, Natalia Gorbokon, Ronald Simon, Martina Kluth, Claudia Hube-Magg, Sarah Minner, Niclas C. Blessin. Tc1 cytotoxic T-cells and Th1 helper T-cell rich tumor microenvironment is a hallmark of MSI colorectal cancers [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 1517.

A Deep Learning Approach to Classification Pneumonia in Thorax Images

An algorithm of automatic learning was developed. That is abke to identify radiographic images with a pneumonia and not pneumonia diagnose based on a data set published in the “Kaggle” platform by the Radiological Society of North America, from which we obtained a set of specific images with their labels that were divided by 70% to train a convolutional neuronal network model consisting of two convolutional layers for the extraction of characteristics in each image, and ends with a classification stage in the training of the model, to conclude with 74% in metrics of accuracy given by test tests with 30% of the data set.

Dissolved Oxygen Forecasting for Lake Erie’s Central Basin Using Hybrid Long Short-Term Memory and Gated Recurrent Unit Networks

Dissolved oxygen (DO) concentration is a pivotal determinant of water quality in freshwater lake ecosystems. However, rapid population growth and discharge of polluted wastewater, urban stormwater runoff, and agricultural non-point source pollution runoff have triggered a significant decline in DO levels in Lake Erie and other freshwater lakes located in populated temperate regions of the globe. Over eleven million people rely on Lake Erie, which has been adversely impacted by anthropogenic stressors resulting in deficient DO concentrations near the bottom of Lake Erie’s Central Basin for extended periods. In the past, hybrid long short-term memory (LSTM) models have been successfully used for the time-series forecasting of water quality in rivers and ponds. However, the prediction errors tend to grow significantly with the forecasting period. Therefore, this research aimed to improve the accuracy of DO forecasting models by taking advantage of Lake Erie’s real-time water quality (water temperature and DO concentration) monitoring network to establish temporal and spatial links between adjacent monitoring stations. We developed hybrid LSTM models that combine LSTM, convolutional neuron network LSTM (CNN-LSTM), hybrid CNN with gated recurrent unit (CNN-GRU) models, and convolutional LSTM (ConvLSTM) to forecast near-bottom DO concentrations in Lake Erie’s Central Basin. These hybrid LSTM models improve their capacity to handle complicated datasets with spatial and temporal variability. These models can serve as accurate and reliable tools for forecasting DO concentrations in freshwater lakes to help environmental protection agencies better access and manage the health of these vital ecosystems. Following analysis of a 21-site Lake Erie dataset for 2020 and 2021, the ConvLSTM model emerged as the most accurate and reliable, boasting an MSE of 0.51 mg/L, MAE of 0.42 mg/L, and an R-squared of 0.95 over the 12 h prediction range. The model foresees future hypoxia in Lake Erie. Notably, the temperature near site 713 holds significance for Central Basin DO forecasting in Lake Erie, as indicated by outcomes derived from the Shapley additive explanations (SHAP).

Quality Testing in Aluminum Die-Casting – A Novel Approach using Acoustic Data in Neural Networks

In quality control of aluminum die casting various processes are used. For example, the density of the parts can be measured, X-ray images or images from the computed tomography are analyzed. All common processes lead to practically usable results. However, the problem arises that none of the processes is suitable for inline quality control due to their time duration and to their costs of hardware. Therefore, a concept for a fast and low-cost quality control process using sound samples is presented here. Sound samples of 240 aluminum castings are recorded and checked for their quality using X-ray images. All parts are divided into the categories "good" without defects, "medium" with air inclusions ("blowholes") and "poor" with cold flow marks. For the processing of the generated sound samples, a Convolutional Neuronal Network was developed. The training of the neural network was performed with both complete and segmented sound samples ("windowing"). The generated models have been evaluated with a test data set consisting of 120 sound samples. The results are very promising. Both models show an accuracy of 95% and 87% percent, respectively. The results show that a new process of acoustic quality control can be realized using a neural network. The model classifies most of the aluminum castings into the correct categories. Keywords: acoustic quality control, aluminum die casting, convolutional neural networks, sound data

Convolutional neuronal network for identifying single-cell-platelet-platelet-aggregates in human whole blood using imaging flow cytometry.

Imaging flow cytometry is an attractive method to investigate individual cells by optical properties. However, imaging flow cytometry applications with clinical relevance are scarce so far. Platelet aggregation naturally occurs during blood coagulation to form a clot. However, aberrant platelet aggregation is associated with cardiovascular disease under steady-state conditions in the blood. Several types of so-called antiplatelet drugs are frequently described to reduce the risk of stroke or cardiovascular diseases. However, an efficient monitoring method is missing to identify the presence and frequency of platelet-platelet aggregates in whole blood on a single cell level. In this work, we employed imaging flow cytometry to identify fluorescently labeled platelets in whole blood with a conditional gating strategy. Images were post-processed and aligned. A convolutional neural network was designed to identify platelet-platelet aggregates of two, three, and more than three platelets, and results were validated against various data set properties. In addition, the neural network excluded erythrocyte-platelet aggregates from the results. Based on the results, a parameter for detecting platelet-platelet aggregates, the weighted platelet aggregation, was developed. If employed on a broad scale with proband and patient samples, our method could aid in building a future diagnostic marker for cardiovascular disease and monitoring parameters to optimize drug prescriptions in such patient groups.

Optimizing olive disease classification through transfer learning with unmanned aerial vehicle imagery

Early detection of diseases in growing olive trees is essential for reducing costs and increasing productivity in this crucial economic activity. The quality and quantity of olive oil depend on the health of the fruit, making accurate and timely information on olive tree diseases critical to monitor growth and anticipate fruit output. The use of unmanned aerial vehicles (UAVs) and deep learning (DL) has made it possible to quickly monitor olive diseases over a large area indeed of limited sampling methods. Moreover, the limited number of research studies on olive disease detection has motivated us to enrich the literature with this work by introducing new disease classes and classification methods for this tree. In this study, we present a UAV system using convolutional neuronal network (CNN) and transfer learning (TL). We constructed an olive disease dataset of 14K images, processed and trained it with various CNN in addition to the proposed MobileNet-TL for improved classification and generalization. The simulation results confirm that this model allows for efficient diseases classification, with a precision accuracy achieving 99% in validation. In summary, TL has a positive impact on MobileNet architecture by improving its performance and reducing the training time for new tasks.

PbS/CsPbBr3 Heterojunction for Broadband Neuromorphic Vision Sensing.

Neuromorphic light sensors with analogue-domain image processing capability hold promise for overcoming the energy efficiency limitations and latency of von Neumann architecture-based vision chips. Recently, metal halide perovskites, with strong light-matter interaction, long carrier diffusion length, and exceptional photoelectric conversion efficiencies, exhibit reconfigurable photoresponsivity due to their intrinsic ion migration effect, which is expected to advance the development of visual sensors. However, suffering from a large bandgap, it is challenging to achieve highly tunable responsivity simultaneously with a wide-spectrum response in perovskites, which will significantly enhance the image recognition accuracy through the machine learning algorithm. Herein, we demonstrate a broadband neuromorphic visual sensor from visible (Vis) to near-infrared (NIR) by coupling all-inorganic metal halide perovskites (CsPbBr3) with narrow-bandgap lead sulfide (PbS). The PbS/CsPbBr3 heterostructure is composed of high-quality single crystals of PbS and CsPbBr3. Interestingly, the ion migration of CsPbBr3 with the implementation of an electric field induces the energy band dynamic bending at the interface of the PbS/CsPbBr3 heterojunction, leading to reversible, multilevel, and linearly tunable photoresponsivity. Furthermore, the reconfigurable and broadband photoresponse in the PbS/CsPbBr3 heterojunction allows convolutional neuronal network processing for pattern recognition and edge enhancements from the Vis to the NIR waveband, suggesting the great potential of the PbS/CsPbBr3 heterostructure in artificial intelligent vision sensing.

A deep learning-based concept for quantitative phase imaging upgrade of bright-field microscope

In this paper, we propose an approach that combines wavefront encoding and convolutional neuronal network (CNN)-based decoding for quantitative phase imaging (QPI). Encoding is realized by defocusing, and decoding by CNN trained on simulated datasets. We have demonstrated that based on the proposed approach of creating the dataset, it is possible to overcome the typical pitfall of CNN learning, such as the shortage of reliable data. In the proposed data flow, CNN training is performed on simulated data, while CNN application is performed on real data. Our approach is benchmarked in real-life experiments with a digital holography approach. Our approach is purely software-based: the QPI upgrade of a bright-field microscope does not require extra optical components such as reference beams or spatial light modulators.

Raising the curtain: Bubble size measurement inside parallel plate electrolyzers

Alongside vertical gas-evolving electrodes, gas bubbles nucleate, grow, coalesce, detach, and ultimately form a bubble curtain, affecting the fluid dynamic, electrochemical reaction, mass transport, and ohmic drop. Due to the small inter-electrode gap in parallel plate electrolyzers, the size determination of detached bubbles inside the bubble curtain close to the electrodes is challenging. In particular, the size determination of detached oxygen bubbles needs to be addressed. We present an optical setup to record images from detached oxygen bubbles inside the bubble curtain of a membrane-separated, parallel plate electrolyzer. The images are postprocessed using an adopted region-based convolutional neuronal network to quantify the bubble sizes. Next, the electrolyte flow velocity, concentration, current density, and catalysts are varied, and the effect on the oxygen bubble size distribution is studied. The median size ranges from 140 to ▪, doubling values from literature measured in artificial electrolyzers.

Deep Reinforcement Learning for Efficient Digital Pap Smear Analysis

In August 2020, the World Health Assembly launched a global initiative to eliminate cervical cancer by 2030, setting three primary targets. One key goal is to achieve a 70% screening coverage rate for cervical cancer, primarily relying on the precise analysis of Papanicolaou (Pap) or digital Pap smears. However, the responsibility of reviewing Pap smear samples to identify potentially cancerous cells primarily falls on pathologists—a task known to be exceptionally challenging and time-consuming. This paper proposes a solution to address the shortage of pathologists for cervical cancer screening. It leverages the OpenAI-GYM API to create a deep reinforcement learning environment utilizing liquid-based Pap smear images. By employing the Proximal Policy Optimization algorithm, autonomous agents navigate Pap smear images, identifying cells with the aid of rewards, penalties, and accumulated experiences. Furthermore, the use of a pre-trained convolutional neuronal network like Res-Net50 enhances the classification of detected cells based on their potential for malignancy. The ultimate goal of this study is to develop a highly efficient, automated Papanicolaou analysis system, ultimately reducing the need for human intervention in regions with limited pathologists.

Influence of CT dose reduction on AI-driven malignancy estimation of incidental pulmonary nodules

ObjectivesThe purpose of this study was to determine the influence of dose reduction on a commercially available lung cancer prediction convolutional neuronal network (LCP-CNN).MethodsCT scans from a cohort provided by the local lung cancer center (n = 218) with confirmed pulmonary malignancies and their corresponding reduced dose simulations (25% and 5% dose) were subjected to the LCP-CNN. The resulting LCP scores (scale 1–10, increasing malignancy risk) and the proportion of correctly classified nodules were compared. The cohort was divided into a low-, medium-, and high-risk group based on the respective LCP scores; shifts between the groups were studied to evaluate the potential impact on nodule management. Two different malignancy risk score thresholds were analyzed: a higher threshold of ≥ 9 (“rule-in” approach) and a lower threshold of > 4 (“rule-out” approach).ResultsIn total, 169 patients with 196 nodules could be included (mean age ± SD, 64.5 ± 9.2 year; 49% females). Mean LCP scores for original, 25% and 5% dose levels were 8.5 ± 1.7, 8.4 ± 1.7 (p > 0.05 vs. original dose) and 8.2 ± 1.9 (p < 0.05 vs. original dose), respectively. The proportion of correctly classified nodules with the “rule-in” approach decreased with simulated dose reduction from 58.2 to 56.1% (p = 0.34) and to 52.0% for the respective dose levels (p = 0.01). For the “rule-out” approach the respective values were 95.9%, 96.4%, and 94.4% (p = 0.12). When reducing the original dose to 25%/5%, eight/twenty-two nodules shifted to a lower, five/seven nodules to a higher malignancy risk group.ConclusionCT dose reduction may affect the analyzed LCP-CNN regarding the classification of pulmonary malignancies and potentially alter pulmonary nodule management.Clinical relevance statementUtilization of a “rule-out” approach with a lower malignancy risk threshold prevents underestimation of the nodule malignancy risk for the analyzed software, especially in high-risk cohorts.Key Points• LCP-CNN may be affected by CT image parameters such as noise resulting from low-dose CT acquisitions.• CT dose reduction can alter pulmonary nodule management recommendations by affecting the outcome of the LCP-CNN.• Utilization of a lower malignancy risk threshold prevents underestimation of pulmonary malignancies in high-risk cohorts.

Automated signal intensity analysis of the spinal cord for detection of degenerative cervical myelopathy — a matched-pair MRI study

PurposeDetection of T2 hyperintensities in suspected degenerative cervical myelopathy (DCM) is done subjectively in clinical practice. To gain objective quantification for dedicated treatment, signal intensity analysis of the spinal cord is purposeful. We investigated fully automated quantification of the T2 signal intensity (T2-SI) of the spinal cord using a high-resolution MRI segmentation.MethodsMatched-pair analysis of prospective acquired cervical 3D T2-weighted sequences of 114 symptomatic patients and 88 healthy volunteers. Cervical spinal cord was segmented automatically through a trained convolutional neuronal network with subsequent T2-SI registration slice-by-slice. Received T2-SI curves were subdivided for each cervical level from C2 to C7. Additionally, all levels were subjectively classified concerning a present T2 hyperintensity. For T2-positive levels, corresponding T2-SI curves were compared to curves of age-matched volunteers at the identical level.ResultsForty-nine patients showed subjective T2 hyperintensities at any level. The corresponding T2-SI curves showed higher signal variabilities reflected by standard deviation (18.51 vs. 7.47 a.u.; p < 0.001) and range (56.09 vs. 24.34 a.u.; p < 0.001) compared to matched controls. Percentage of the range from the mean absolute T2-SI per cervical level, introduced as “T2 myelopathy index” (T2-MI), was correspondingly significantly higher in T2-positive segments (23.99% vs. 10.85%; p < 0.001). ROC analysis indicated excellent differentiation for all three parameters (AUC 0.865–0.920).ConclusionThis fully automated T2-SI quantification of the spinal cord revealed significantly increased signal variability for DCM patients compared to healthy volunteers. This innovative procedure and the applied parameters showed sufficient diagnostic accuracy, potentially diagnosing radiological DCM more objective to optimize treatment recommendation.Trial registrationDRKS00012962 (17.01.2018) and DRKS00017351 (28.05.2019)

Agreement and accuracy of fully automated morphometric femorotibial cartilage analysis in radiographic knee osteoarthritis

ObjectiveTo examine the performance of automated convolutional neuronal network cartilage segmentation in knees with radiographic osteoarthritis (ROA), and its dependence on the OA status of the training set and MRI sequences. DesignWe studied 122 ROA and 92 healthy reference cohort (HRC) knees from the Osteoarthritis Initiative. All knees had expert manual segmentation of the femorotibial cartilages based on coronal FLASH and sagittal DESS MRI. Two U-net convolutional neural networks were trained on 86/50 ROA/HRC knees, validated on 18/21, and tested on 18/21. ResultsOf 122 ROA knees, 43 (35%) were KLG2, 41 (34%) KLG3, and 38 (31%) KLG4. In the ROA test set, Dice Similarity Coefficients (DSCs) were 0.86/0.86 (FLASH/DESS) for the algorithm trained on ROA, and 0.82/0.82 for that trained on HRC knees. In the HRC test set, mean DSCs of 0.91/0.90 were observed with FLASH/DESS, both with the HRC- and with the ROA-trained algorithm. Cartilage thickness computation from automated segmentation in the FLASH ROA test set showed a correlation of r = 0.94 with manual segmentation for the ROA-trained, and of r = 0.89 for the HRC-trained algorithm. In the FLASH HRC test set, it exhibited a correlation of r = 0.96 for the HRC-trained and r = 0.88 for the ROA-trained algorithm. Results were similar between the DESS and FLASH, but were less accurate for KLG4 than for KLG2/3 knees. ConclusionsAn automated algorithm trained on ROA knees was able to accurately segment and compute cartilage thickness on FLASH and DESS MRI, in both ROA and healthy knees, whereas an algorithm trained on healthy knees did not perform as well in ROA knees.

Dynamic evolution of brain structural patterns in liver transplantation recipients: a longitudinal study based on 3D convolutional neuronal network model.

To evaluate the dynamic evolution process of overall brain health in liver transplantation (LT) recipients, we employed a deep learning-based neuroanatomic biomarker to measure longitudinal changes of brain structural patterns before and 1, 3, and 6months after surgery. Because of the ability to capture patterns across all voxels from a brain scan, the brain age prediction method was adopted. We constructed a 3D-CNN model through T1-weighted MRI of 3609 healthy individuals from 8 public datasets and further applied it to a local dataset of 60 LT recipients and 134 controls. The predicted age difference (PAD) was calculated to estimate brain changes before and after LT, and the network occlusion sensitivity analysis was used to determine the importance of each network in age prediction. The PAD of patients with cirrhosis increased markedly at baseline (+ 5.74years) and continued to increase within one month after LT (+ 9.18years). After that, the brain age began to decrease gradually, but it was still higher than the chronological age. The PAD values of the OHE subgroup were higher than those of the no-OHE, and the discrepancy was more obvious at 1-month post-LT. High-level cognition-related networks were more important in predicting the brain age of patients with cirrhosis at baseline, while the importance of primary sensory networks increased temporarily within 6-month post-LT. The brain structural patterns of LT recipients showed inverted U-shaped dynamic change in the early stage after transplantation, and the change in primary sensory networks may be the main contributor. • The recipients' brain structural pattern showed an inverted U-shaped dynamic change after LT. • The patients' brain aging aggravated within 1month after surgery, and the subset of patients with a history of OHE was particularly affected. • The change of primary sensory networks is the main contributor to the change in brain structural patterns.

A deep learning-based histopathology classifier for Focal Cortical Dysplasia

A light microscopy-based histopathology diagnosis of human brain specimens obtained from epilepsy surgery remains the gold standard to confirm the underlying cause of a patient’s focal epilepsy and further inform postsurgical patient management. The differential diagnosis of neocortical specimens in the realm of epilepsy surgery remains, however, challenging. Herein, we developed an open access, deep learning-based classifier to histopathologically assess whole slide microscopy images (WSI) and to automatically recognize various subtypes of Focal Cortical Dysplasia (FCD), according to the ILAE consensus classification update of 2022. We trained a convolutional neuronal network (CNN) with fully digitalized WSI of hematoxylin–eosin stainings obtained from 125 patients covering the spectrum of mild malformation of cortical development (mMCD), mMCD with oligodendroglial hyperplasia in epilepsy (MOGHE), FCD ILAE Type 1a, 2a and 2b using 414 formalin-fixed and paraffin-embedded archival tissue blocks. An additional series of 198 postmortem tissue blocks from 59 patients without neurological disorders served as control to train the CNN for homotypic frontal, temporal and occipital areas and heterotypic Brodmann areas 4 and 17, entorhinal cortex and dentate gyrus. Special stains and immunohistochemical reactions were used to comprehensively annotate the region of interest. We then programmed a novel tile extraction pipeline and graphical dashboard to visualize all areas on the WSI recognized by the CNN. Our deep learning-based classifier is able to compute 1000 × 1000 µm large tiles and recognizes 25 anatomical regions and FCD categories with an accuracy of 98.8% (F1 score = 0.82). Microscopic review of regions predicted by the network confirmed these results. This deep learning-based classifier will be made available as online web application to support the differential histopathology diagnosis in neocortical human brain specimens obtained from epilepsy surgery. It will also serve as blueprint to build a digital histopathology slide suite addressing all major brain diseases encountered in patients with surgically amenable focal epilepsy.

Improved fully convolutional neuron networks on small retinal vessel segmentation using local phase as attention.

Retinal images have been proven significant in diagnosing multiple diseases such as diabetes, glaucoma, and hypertension. Retinal vessel segmentation is crucial for the quantitative analysis of retinal images. However, current methods mainly concentrate on the segmentation performance of overall retinal vessel structures. The small vessels do not receive enough attention due to their small percentage in the full retinal images. Small retinal vessels are much more sensitive to the blood circulation system and have great significance in the early diagnosis and warning of various diseases. This paper combined two unsupervised methods, local phase congruency (LPC) and orientation scores (OS), with a deep learning network based on the U-Net as attention. And we proposed the U-Net using local phase congruency and orientation scores (UN-LPCOS), which showed a remarkable ability to identify and segment small retinal vessels. A new metric called sensitivity on a small ship (Sesv ) was also proposed to evaluate the methods' performance on the small vessel segmentation. Our strategy was validated on both the DRIVE dataset and the data from Maastricht Study and achieved outstanding segmentation performance on both the overall vessel structure and small vessels.

CNN-based evaluation of bone density improves diagnostic performance to detect osteopenia and osteoporosis in patients with non-contrast chest CT examinations

PurposeAs osteoporosis is still underdiagnosed by clinicians and radiologists, the aim of the present study was to assess the performance of an Artificial intelligence (AI)-based Convolutional Neuronal Network (CNN)-Algorithm for the detection of low bone density on routine non-contrast chest CT in comparison to clinical reports using DEXA scans as reference. MethodThis retrospective cross-sectional study included patients who underwent non-contrast chest CT and DEXA between April 2018 and June 2018 (n = 109, 19 men, mean age: 67.7 years). CT studies were evaluated for thoracic vertebral bone pathologies using a CNN-Algorithm, which calculates the attenuation profile of the spine. The content of the radiological reports was evaluated for the description of osteoporosis or osteopenia. DEXA was used as the reference standard. To estimate correlation the Spearman test was used and the comparison of the different groups was performed using the Wilcoxon rank sum test. Diagnostic was evaluated by performing a receiver operating characteristic curve analysis. ResultsThe DEXA examination revealed normal bone density in 42 patients, while 49 patients had osteopenia and 7 osteoporosis. There was a statistically significant correlation between the mean CNN-based attenuation of the thoracic spine and the bone density measured on the DEXA in the hip (r = 0.51, p < 0.001) and lumbar spine (r = 0.34, p = 0.01). The mean attenuation was significantly higher in patients with normal bone density (172 ± 44.5 HU) compared to those with osteopenia or osteoporosis (125.2 ± 33.8 HU), (p < 0.0001). Diagnostic performance in distinguishing normal from abnormal bone density was higher using the CNN-based vertebral attenuation (accuracy 0.75, sensitivity: 0.93, specificity: 0.61) compared to clinical reports (accuracy 0.51, sensitivity: 0.14, specificity: 0.53). ConclusionCNN-based evaluation of bone density may provide additional value over standard clinical reports for the detection of osteopenia and osteoporosis in patients undergoing routine non-contrast chest CT scans. This additional value could improve identification of fracture risk and subsequent treatment.

A Reconfigurable CNN-Based Accelerator Design for Fast and Energy-Efficient Object Detection System on Mobile FPGA

In limited-resource edge computing circumstances such as on mobile devices, IoT devices, and electric vehicles, the energy-efficient optimized convolutional neural network (CNN) accelerator implemented on mobile Field Programmable Gate Array (FPGA) is becoming more attractive due to its high accuracy and scalability. In recent days, mobile FPGAs such as the Xilinx PYNQ-Z1/Z2 and Ultra96, definitely have the advantage of scalability and flexibility for the implementation of deep learning algorithm-based object detection applications. It is also suitable for battery-powered systems, especially for drones and electric vehicles, to achieve energy efficiency in terms of power consumption and size aspect. However, it has the low and limited performance to achieve real-time processing. In this article, optimizing the accelerator design flow in the register-transfer level (RTL) will be introduced to achieve fast programming speed by applying low-power techniques on FPGA accelerator implementation. In general, most accelerator optimization techniques are conducted on the system level on the FPGA. In this article, we propose the reconfigurable accelerator design for a CNN-based object detection system on the register-transfer level on mobile FPGA. Furthermore, we present RTL optimization design techniques that will be applied such as various types of clock gating techniques to eliminate residual signals and to deactivate the unnecessarily active block. Based on the analysis of the CNN-based object detection architecture, we analyze and classify the common computing operation components from the Convolutional Neuron Network, such as multipliers and adders. We implement a multiplier/adder unit to a universal computing unit and modularize it to be suitable for a hierarchical structure of RTL code. The proposed system design was tested with Resnet-20 which has 23 layers and it was trained with the dataset, CIFAR-10 which provides a test set of 10,000 images in several formats, and the weight data we used for this experiment was provided from Tensil. Experimental results show that the proposed design process improves the power efficient consumption, hardware utilization, and throughput by 16%, up to 58%, and 15%, respectively.

Convolutional neural network optimisation to enhance ESPI fringe visibility

The use of convolutional neuronal networks (CNN) for the treatment of interferometric fringes has been introduced in recent years. In this paper, we optimize and build a CNN model, based U-NET architecture, to maximize its performance processing electronic speckle interferometry fringes (ESPI). The proposed approach is based on quick and light trainings to select the architecture parameters (network depth and kernel sizes) to maximize the performance of the neural network improving the visibility of ESPI images. To measure the performance, the structural similarity index (SSMI) will be the lead indicator, and the need for large datasets to train neural networks, unavailable for ESPI images, forces the use of a simulated ESPI image dataset along the process. This dataset is computed using Zernike polynomials to simulate local surface deformations in the specimen under test and simulated true speckle fields for the reference and object field involved in ESPI techniques.