Digital Pipeline Research Articles

Digitisation of electrocardiographic images enable artificial intelligence-based mortality prediction

Abstract Background Artificial intelligence-based electrocardiogram (AI-ECG) models have shown excellent performance in a vast number of diagnostic and prognostic tasks. One of such tasks, which is critical for patient risk stratification and clinical intervention, is the prediction of adverse events, including death. Although a plethora of existing AI-ECG models have shown their accuracy in mortality prediction, their development and utilization currently rely on digital ECG signals for training and application. Purpose Many hospitals around the world have a significant number of ECGs stored in image formats or printed as paper ECGs. To address this problem, we developed and tested an ECG image digitisation pipeline to extract digital traces from ECG images and compare the performance of AI-ECG models for mortality prediction, under both natively digital and digitised signal formats. Methods A secondary care dataset of over 1 million ECGs (1,163,401 ECGs; 189,539 patients) was used for this analysis, in which both a natively digital signal and a PDF image was available. The ECG digitisation pipeline comprised of coloured PDF image inputs (2200 x 1700), initially processed to extract the ECG traces and grid features. ECG signals were extracted from the traces after cropping/restoration of the images and scaled appropriately based on the detected grid (2.5-sec strips per lead, 0.5 mV per grid block). Subsequently, both digital and digitised signals were identically pre-processed in order to acquire the ECG data fed into the AI model. In particular, a previously successful convolutional neural network architecture with a discrete-time survival loss function was used for mortality prediction. The model was trained under multiple configurations, using either the digital or the digitised ECGs as training/testing datasets (Figure 1). ECGs were split by patient ID to ensure cross-subject generalization with a training/validation/testing split of 50%/10%/40% ratio, respectively. Finally, the models were evaluated by the c-index. Results A comparison of the digital and digitised ECGs on the respective 2.5-sec strips available in both formats revealed a mean absolute error of 0.014 (CI: 0.07 – 0.021) and a Pearson’s R of 0.98 (CI: 0.96 – 1.00) between the signals, demonstrating an accurate ECG digitisation (Figure 2 shows overlapping natively digital (black) and digitised (red) traces). The performance of the digitisation was also evident in the performance of the AI-ECG mortality prediction task, with the model achieving c-indices of 0.775 (CI: 0.774 – 0.776) and 0.772 (CI: 0.771 – 0.774) when trained and tested on digital and digitised signals, respectively. Conclusion We described the first digitisation-based AI-ECG model for mortality prediction. Our results demonstrate the potential for clinical settings that lack the digital ECG infrastructure, to develop and deploy AI models using ECG image formats.Figure 1Figure 2

Research on the Coupled Representation of Model and Data Integration Guided by Metadata Sets

Abstract. Aiming at the problems of islands, redundancy and sharing difficulties of spatial and temporal business models and data in geospatial and spatial business systems, this paper proposes an integrated and coupled expression method of models and data guided by metadata sets, based on the research on core metadata of information resources and component-based encapsulation. First of all, this method classifies the model and data features, and constructs the metadata description and expression framework dominated by feature meta-components; then, establishes metadata sets through the expansion of the metadata description and expression framework; then, constructs the correlation between the model and the data through the use of the established metadata sets by means of the component-based encapsulation method, so as to complete the integrated coupling of the model and the data; and finally, realises the integrated coupling expression of the model and the data in the digital pipeline business system. The results show that this method integrates the expression of digital pipeline element models and data, solves the problems of silos, redundancy and conflicts of heterogeneous models and data from multiple sources, ensures the integrity of the attributes of the models and data elements, and promotes the structured expression and application of the models and data.

Decoding urban evolution: A parametric reconstruction of the Old City of Aleppo between the mamluk and the post-war city

The Old City of Aleppo suffered extensive urban damage during the war in Syria. This study showcases the implementation of digital pipelines in visualizing and studying the history and evolution of two districts within the Old City. The aim of this study was to develop a parametric workflow that would allow for the reconstruction of a section of Mamluk Aleppo. This reconstruction was based on both cartographical and historical inputs. A comprehensive digital field survey was conducted using laser scanning and aerial photogrammetry techniques. The resulting point cloud data is then compared to the historical parametric model, and 3D urban changes to the studied area were identified.

AI-Accelerated Digitisation of Insect Collections: The next generation of Angled Label Image Capture Equipment (ALICE)

The digitisation of natural science specimens is a shared ambition of many of the largest collections, but the scale of these collections, estimated at at least 1.1 billion specimens (Johnson et al. 2023), continues to challenge even the most resource-rich organisations. The Natural History Museum, London (NHM) has been pioneering work to accelerate the digitisation of its 80 million specimens. Since the inception of the NHM Digital Collection Programme in 2014, more than 5.5 million specimen records have been made digitally accessible. This has enabled the museum to deliver a tenfold increase in digitisation, compared to when rates were first measured by the NHM in 2008. Even with this investment, it will take circa 150 years to digitise its remaining collections, leading the museum to pursue technology-led solutions alongside increased funding to deliver the next increase in digitisation rate. Insects comprise approximately half of all described species and, at the NHM, represent more than one-third (c. 30 million specimens) of the NHM’s overall collection. Their most common preservation method, attached to a pin alongside a series of labels with metadata, makes insect specimens challenging to digitise. Early Artificial Intelligence (AI)-led innovations (Price et al. 2018) resulted in the development of ALICE, the museum's Angled Label Image Capture Equipment, in which a pinned specimen is placed inside a multi-camera setup, which captures a series of partial views of a specimen and its labels. Centred around the pin, these images can be digitally combined and reconstructed, using the accompanying ALICE software, to provide a clean image of each label. To do this, a Convolutional Neural Network (CNN) model is incorporated, to locate all labels within the images. This is followed by various image processing tools to transform the labels into a two-dimensional viewpoint, align the associated label images together, and merge them into one label. This allows users to manually, or computationally (e.g., using Optical Character Recognition [OCR] tools) extract label data from the processed label images (Salili-James et al. 2022). With the ALICE setup, a user might average imaging 800 digitised specimens per day, and exceptionally, up to 1,300. This compares with an average of 250 specimens or fewer daily, using more traditional methods involving separating the labels and photographing them off of the pin. Despite this, our original version of ALICE was only suited to a small subset of the collection. In situations when the specimen is very large, there are too many labels, or these labels are too close together, ALICE fails (Dupont and Price 2019). Using a combination of updated AI processing tools, we hereby present ALICE version 2. This new version of ALICE provides faster rates, improved software accuracy, and a more streamlined pipeline. It includes the following updates: Hardware: after conducting various tests, we have optimised the camera setup. Further hardware updates include a Light-Emitting Diode (LED) ring light, as well as modifications to the camera mounting. Software: our latest software incorporates machine learning and other computer vision tools to segment labels from ALICE images and stitch them together more quickly and with a higher level of accuracy, significantly reducing the image processing failure rate. These processed label images can be combined with the latest OCR tools for automatic transcription and data segmentation. Buildkit: we aim to provide a toolkit that any individual or institution can incorporate into their digitisation pipeline. This includes hardware instructions, an extensive guide detailing the pipeline, and new software code accessible via Github. Hardware: after conducting various tests, we have optimised the camera setup. Further hardware updates include a Light-Emitting Diode (LED) ring light, as well as modifications to the camera mounting. Software: our latest software incorporates machine learning and other computer vision tools to segment labels from ALICE images and stitch them together more quickly and with a higher level of accuracy, significantly reducing the image processing failure rate. These processed label images can be combined with the latest OCR tools for automatic transcription and data segmentation. Buildkit: we aim to provide a toolkit that any individual or institution can incorporate into their digitisation pipeline. This includes hardware instructions, an extensive guide detailing the pipeline, and new software code accessible via Github. We provide test data and workflows to demonstrate the potential of ALICE version 2 as an effective, accessible, and cost-saving solution to digitising pinned insect specimens. We also describe potential modifications, enabling it to work with other types of specimens.

Congo Red Staining in Digital Pathology: The Streamlined Pipeline for Amyloid Detection Through Congo Red Fluorescence Digital Analysis

Renal amyloidosis is a rare condition caused by the progressive accumulation of misfolded proteins within glomeruli, vessels, and interstitium, causing functional decline and requiring prompt treatment due to its significant morbidity and mortality. Congo red (CR) stain on renal biopsy samples is the gold standard for diagnosis, but the need for polarized light is limiting the digitization of this nephropathology field. This study explores the feasibility and reliability of CR fluorescence on virtual slides (CRFvs) in evaluating the diagnostic accuracy and proposing an automated digital pipeline for its assessment. Whole-slide images from 154 renal biopsies with CR were scanned through a Texas red fluorescence filter (NanoZoomer S60, Hamamatsu) at the digital Nephropathology Center of the Istituto di Ricovero e Cura a Carattere Scientifico San Gerardo, Monza, Italy, and evaluated double-blinded for the detection and quantification through the amyloid score and a custom ImageJ pipeline was built to automatically detect amyloid-containing regions. Interobserver agreement for CRFvs was optimal (k = 0.90; 95% CI, 0.81-0.98), with even better concordance when consensus-based CRFvs evaluation was compared to the standard CR birefringence (BR) (k = 0.98; 95% CI, 0.93-1). Excellent performance was achieved in the assessment of amyloid score overall by CRFvs (weighted k = 0.70; 95% CI, 0.08-1), especially within the interstitium (weighted k = 0.60; 95% CI, 0.35-0.84), overcoming the misinterpretation of interstitial and capsular collagen BR. The application of an automated digital pathology pipeline (Streamlined Pipeline for Amyloid detection through CR fluorescence Digital Analysis, SPADA) further increased the performance of pathologists, leading to a complete concordance with the standard BR. This study represents an initial step in the validation of CRFvs, demonstrating its general reliability in a digital nephropathology center. The computational method used in this study has the potential to facilitate the integration of spatial omics and artificial intelligence tools for the diagnosis of amyloidosis, streamlining its detection process.

Impact of tessellation surface model generation process on acoustical tire features

Due to the electrification of vehicles, the focus of vehicle noise is shifting to the tires. Since tire noise is dependent of the tread pattern, methods to characterize tread patterns are required. This research investigates the effects of different processes of tire digitization on the identification of tread pattern features. An open source digitization pipeline is built to extract a tessellation surface model. This model is compared to one, generated with a commercial photogrammetric system. The comparison is based on three acoustically relevant tire features. Fully automated algorithms are introduced to extract these features from any 3D tire tessellation model. The research finds that although the resolution of the mesh surface of the proposed model is lower, feature recognition is not affected by this change. This paves the way for more accessible models that accelerate the statistical coupling of tire tread characteristics and their acoustic, handling or braking behavior.

WEB PLATFORMS FOR CULTURAL HERITAGE MANAGEMENT: THE PARCO ARCHEOLOGICO DEL COLOSSEO CASE STUDY

Abstract. This paper describes the digitization test of Fonte Giuturna (Giuturna spring) in the Roman Forum area, from survey to data management through the in-use monitoring system, the WebApp SyPEAH of the Parco Archeologico del Colosseo. The location of Giuturna Spring, characterized by the presence of heterogeneous archaeological remains from different ages, was surveyed in May 2022 as part of a research project that aimed to superintend the entire Cultural Heritage digitization pipeline to provide the Archaeological Park Administration the digitization guidelines as a tool to standardize future surveys and data deliveries.Inspired by the desire to build a system of protection and conservation at the service of sustainable exploitation, SyPEAH is a web platform based on open-source modules designed to manage archaeological records with a WebGIS approach. It supports the use of several 3D data formats, including point clouds. The paper focuses on the web platform, describing the web app’s main features, especially in terms of point cloud data management. Moreover, possible future development of the platform intended to implement usability for single archaeological objects is described.

Automated and data-driven plate computation for presurgical cleft lip and palate treatment

PurposePresurgical orthopedic plates are widely used for the treatment of cleft lip and palate, which is the most common craniofacial birth defect. For the traditional plate fabrication, an impression is taken under airway-endangering conditions, which recent digital alternatives overcome via intraoral scanners. However, these alternatives demand proficiency in 3D modeling software in addition to the generally required clinical knowledge of plate design.MethodsWe address these limitations with a data-driven and fully automated digital pipeline, endowed with a graphical user interface. The pipeline adopts a deep learning model to landmark raw intraoral scans of arbitrary mesh topology and orientation, which guides the nonrigid surface registration subsequently employed to segment the scans. The plates that are individually fit to these segmented scans are 3D-printable and offer optional customization.ResultsWith the distance to the alveolar ridges closely centered around the targeted 0.1 mm, our pipeline computes tightly fitting plates in less than 3 min. The plates were approved in 12 out of 12 cases by two cleft care professionals in a printed-model-based evaluation. Moreover, since the pipeline was implemented in clinical routine in two hospitals, 19 patients have been undergoing treatment utilizing our automated designs.ConclusionThe results demonstrate that our automated pipeline meets the high precision requirements of the medical setting employed in cleft lip and palate care while substantially reducing the design time and required clinical expertise, which could facilitate access to this presurgical treatment, especially in low-income countries.

Digitization of 20 years of decametric observations of the Sun and Jupiter in Nançay between 1970 and 1990

Since 1970, decametric observations of the Sun and Jupiter have been acquired on a daily basis from the radio observatory of Nançay, France. The resulting database now forms the worldwide largest set of low frequency observations of the Solar Corona between 10 and 100 MHz and of the Jovian magnetosphere between 10 and 40 MHz. The Nançay Decameter Interferometer (NDI), a two-antenna interferometer, acquired pioneer observations from 1970 to 1978, using analog radio receivers tested on the Arecibo giant radiotelescope. The large Nançay Decameter array (NDA), a phased array of 144 helical antennas, was then constructed in the late 1970s as a ground-based support to the Voyager mission and started routine observations in January 1978 in replacement of the NDI, using similar spectrometers. The NDA is still in operation today and its observations, recorded in a digital format since 1990, are public and daily made accessible to the heliospheric and magnetospheric communities at https://www.obs-nancay.fr/reseau-decametrique/. Before 1990, though, the Nançay decametric observations were recorded on analog devices, such as 35-mm films, magnetic tapes and other paper supports, physically archived at the Observatory of Paris, on its site of the Observatory of Meudon, where they could only be accessed on demand. The NDA team recently undertook a multi-year effort to inventory all the Nançay decametric archives, to refurbish and digitize them in order to preserve and promote this historical data collection, most notably by extending the current public database over the 1970–1990 time interval. We digitized the full collection of 35-mm films, corresponding to a total film length of ∼45.5 km, with a specific high resolution scanner. Each roll was scanned by a series of pixel-to-pixel contiguous images recorded in TIF format with adequate metadata and naming convention. The final ∼40 Tb data volume was archived at Paris Astronomical Data Centre and the physical films were refurbished to be archived in a perennial way. We also digitized the associated observations sheets. Finally, we took advantage of this project to also digitize a sample of 35-mm films extracted from four other historical solar data collections hosted at the Observatory of Paris and dealing with two distinct Nançay radiotelescopes and two Lyot heliographs located at the Observatories of Meudon and Haute-Provence. The digitized, documented, Nançay decametric archives have been publicly released to the community in 2020 at http://archives-decametriques.obspm.fr. After having described the instruments, the data collections and the digitization pipeline, this articles ends with a few scientific case studies, focussing on the search for Jupiter-satellite decametric emissions. For instance, the digitized NDA archives suggest the presence of Gan-A and B emissions, while these were first reported in 2018. This work is a concrete example of how historical astronomical measurements can be preserved and archived along the IAU recommendation, to the benefit of the astronomical community.

Development of a mixed reality method for underground pipelines in digital mechanics experiments

The underground pipeline network (UPN) is an essential underground structure and infrastructure. Its full-cycle digital twin system is an important part of the smart city. However, traditional information transmission between humans and computers lacks understanding and interaction in the digital twin experiments. From the perspective of perception, a space interaction system based on mixed reality (MR) technology is used to solve the problem of interaction for traditional mechanical experiments. Firstly, during the design phase, Building Information Modeling (BIM) is used to integrate models and information storage, including physical geometry data and material information. Secondly, the analytical algorithm is constructed for sensors by data filtering and material mechanics theory; based on sensors communication, C#/C++, and Socket to develop human–computer interactive methods through Game Engines and Mixed Reality Toolkits (MRTK). Finally, based on comparing the mechanical parameters, the testing of digital pipeline experiments by spatial anchors. Twenty-four experimenters are selected to test this method. The advantages and prospects of spatial interaction with MR are summarized through qualitative and quantitative analysis. This study realizes the data perception of the UPN and maps it to the real environment, which can provide a technical reference for engineering digital experiments.

Public Financial Services Schemes for the Financial Inclusion of People under the BJP led NDA Central Government in India

The main objective of financial inclusion is to extend financial services to the large hitherto un-served population of India country to unlock its growth potential. The BJP led NDA central government under the Prime Ministership of Mr. Narendra Modi has initiated a large scheme of National Mission for Financial Inclusion (NMFI), namely, Pradhan Mantri Jan Dhan Yojana (PMJDY) in August, 2014 to provide universal banking services for every unbanked household based on the guiding principles of banking the unbanked, securing the unsecured, funding the unfunded and serving un-served and under-served areas. A digital pipeline has been laid for the implementation of PMJDY through linking of Jan-Dhan account with mobile and Aadhaar [Jan Dhan-Aadhaar-Mobile (JAM)]. With a view to further deepening the financial inclusion interventions in the country, PMJDY has been extended beyond 14.8.2018 with the focus on opening of accounts shifting from “every household” to “every unbanked adult”. The moto of financial inclusion is from Jandhan to Jansuraksha. Apart from this, the Modi government had also initiated a very good number of other schemes under its public financial services for the purpose of financial inclusion of financially excluded people in India. The success of these schemes were tremendous.

Abstract 464: AI-powered segmentation and analysis of nuclei morphology predicts genomic and clinical markers in multiple cancer types

Abstract Morphological features of cancer cell nuclei are linked to gene expression signatures and genomic alterations. In addition, pathologists have leveraged nuclear morphology as diagnostic and prognostic markers. To enable the use of nuclear morphology in digital pathology, we developed a pan-tissue, deep-learning-based digital pathology pipeline for exhaustive nucleus detection, instance segmentation, and classification. We collected > 29,000 manual nucleus annotations from hematoxylin and eosin (H&E)-stained pathology images from 21 tumor types at 40x and 20x magnification from The Cancer Genome Atlas (TCGA) project, as well as a proprietary set of H&E-stained tissue biopsies of skin, liver non-alcoholic steatohepatitis (NASH), colon inflammatory bowel disease (IBD), and kidney lupus. Annotations were used to train an object detection and segmentation model for identifying nuclei. Application of the model to held-out test data, including held-out tissue types, demonstrated performance comparable to state-of-the-art models described in the literature (mean Dice score = 0.80, aggregated Jaccard index = 0.60). We deployed our model to segment nuclei in H&E slides from the breast cancer (BRCA, N = 941) and prostate adenocarcinoma (PRAD, N = 457) TCGA cohorts. We extracted interpretable features describing the shape (circularity, eccentricity), size, staining intensity (mean and standard deviation), and texture of each nucleus. Nuclei were assigned as cancer or other cell types using separately trained convolutional neural networks for BRCA and PRAD. We used the mean and standard deviation of each feature sampled from a random subset of cancer nuclei to summarize the nuclear morphology on each slide (mean (range) = 10,068 (5,981-10,452) cancer cells from each BRCA slide; mean (range) = 10,053 (5,029-10,495) cancer cells from each PRAD slide). We used nuclear features to construct random forest classification models for predicting markers of genomic instability and prognosis: whole-genome doubling (WGD) and homologous recombination deficiency (HRD) status separately in BRCA and PRAD, HER2 subtype in BRCA, and Gleason grade in PRAD. Nuclear features were predictive of WGD (area under the receiver operating characteristic curve (AUROC) = 0.78 BRCA, = 0.69 PRAD) and binarized HRD status (AUROC = 0.65 BRCA, = 0.68 PRAD) on held-out test sets. Nuclear features were predictive of HER2-enriched breast cancer vs. other molecular subtypes (AUROC = 0.72), and distinguished between low risk (6) and moderate/high risk (7-10) Gleason grade in PRAD (AUROC = 0.72). In summary, we present a powerful pan-tissue approach for nucleus segmentation and featurization, which enables the construction of predictive models and the identification of features linking nuclear morphology with clinically-relevant prognostic biomarkers across multiple cancer types. Citation Format: John Abel, Suyog Jain, Deepta Rajan, Ken Leidal, Harshith Padigela, Aaditya Prakash, Jake Conway, Michael Nercessian, Christian Kirkup, Robert Egger, Ben Trotter, Andrew Beck, Ilan Wapinski, Michael G. Drage, Limin Yu, Amaro Taylor-Weiner. AI-powered segmentation and analysis of nuclei morphology predicts genomic and clinical markers in multiple cancer types [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 464.

Arteriolar dysgenesis in ischemic-regenerating skeletal muscle revealed by automated micromorphometry, computational modeling, and perfusion analysis.

Rebuilding the local vasculature is central to restoring the health of muscles subjected to ischemic injury. Arteriogenesis yields remodeled collateral arteries that circumvent the obstruction, and angiogenesis produces capillaries to perfuse the regenerating myofibers. However, the vital intervening network of arterioles that feed the regenerated capillaries is poorly understood and is an investigative challenge. We used machine learning and automated micromorphometry to quantify the arteriolar landscape in distal hindlimb muscles in mice that have regenerated after femoral artery excision. Assessment of 1,546 arteriolar sections revealed a striking (>2-fold) increase in arteriolar density in regenerated muscle 14 and 28 days after ischemic injury. Lumen caliber was initially similar to that of control arterioles but after 4 wk lumen area was reduced by 46%. In addition, the critical smooth muscle layer was attenuated throughout the arteriolar network, across a 150- to 5-µm diameter range. To understand the consequences of the reshaped distal hindlimb arterioles, we undertook computational flow modeling, which revealed blunted flow augmentation. Moreover, impaired flow reserve was confirmed in vivo by laser-Doppler analyses of flow in response to directly applied sodium nitroprusside. Thus, in hindlimb muscles regenerating after ischemic injury, the arteriolar network is amplified, inwardly remodels, and is diffusely undermuscularized. These defects and the associated flow restraints could contribute to the deleterious course of peripheral artery disease and merit attention when considering therapeutic innovations.NEW & NOTEWORTHY We report a digital pipeline for interrogating the landscape of arterioles in mouse skeletal muscle, using machine learning and automated micromorphometry. This revealed that in muscle regenerating after ischemic injury, the arteriolar density is increased but lumen caliber and smooth muscle content are reduced. Computational modeling and experimental validation reveal this arteriolar network to be functionally compromised, with diminished microvascular flow reserve.

Data-Driven Convolutional Model for Digital Color Image Demosaicing

Modern digital cameras use specific arrangement of Color Filter Array to sample light wavelength corresponding to visible colors. The most common Color Filter Array is the Bayer filter that samples only one color per pixel. To recover the full resolution image, an interpolation algorithm can be used. This process is called demosaicing and it is one of the first processing stages of a digital imaging pipeline. We introduce a novel data-driven model for demosaicing that takes into account the different requirements for reconstruction of the image Luma and Chrominance channels. The final model is a parallel composition of two reconstruction networks with individual architecture and trained with distinct loss functions. In order to solve the overfitting problem, we prepared a dataset that contains groups of patches that share common chromatic and spectral characteristics. We reported the reconstruction error on noise-free images and measured the effect of random noise and quantization noise in the demosaicing reconstruction. To test our model performance, we implemented the network on NVIDIA Jetson Nano, obtaining an end-to-end running time of less than one second for a full frame 12 MPixel image.

Contact modeling from images using cut finite element solvers

This paper presents a robust digital pipeline from CT images to the simulation of contact between multiple bodies. The proposed strategy relies on a recently developed immersed finite element algorithm that is capable of simulating unilateral contact between solids without meshing (Claus and Kerfriden in Int J Numer Methods Eng 113(6):938–966, 2018). It was shown that such an approach reduces the difficulties associated with the digital flow of information from analytically defined geometries to mechanical simulations. We now propose to extend our approach to include geometries, which are not defined mathematically but instead are obtained from images, and encoded in 3D arrays of voxels. This paper introduces two novel elements. Firstly, we reformulate our contact algorithm into an extension of an augmented Lagrangian CutFEM algorithm. Secondly, we develop an efficient algorithm to convert the surface data generated by standard segmentation tools used in medical imaging into level-set functions. These two elements give rise to a robust digital pipeline with minimum user intervention. We demonstrate the capabilities of our algorithm on a hip joint geometry with contact between the femur and the hip bone.

Development and virtual validation of a novel digital workflow to rehabilitate palatal defects by using smartphone-integrated stereophotogrammetry (SPINS)

Palatal defects are rehabilitated by fabricating maxillofacial prostheses called obturators. The treatment incorporates taking deviously unpredictable impressions to facsimile the palatal defects into plaster casts for obturator fabrication in the dental laboratory. The casts are then digitally stored using expensive hardware to prevent physical damage or data loss and, when required, future obturators are digitally designed, and 3D printed. Our objective was to construct and validate an economic in-house smartphone-integrated stereophotogrammetry (SPINS) 3D scanner and to evaluate its accuracy in designing prosthetics using open source/free (OS/F) digital pipeline. Palatal defect models were scanned using SPINS and its accuracy was compared against the standard laser scanner for virtual area and volumetric parameters. SPINS derived 3D models were then used to design obturators by using (OS/F) software. The resultant obturators were virtually compared against standard medical software designs. There were no significant differences in any of the virtual parameters when evaluating the accuracy of both SPINS, as well as OS/F derived obturators. However, limitations in the design process resulted in minimal dissimilarities. With further improvements, SPINS based prosthetic rehabilitation could create a viable, low cost method for rural and developing health services to embrace maxillofacial record keeping and digitised prosthetic rehabilitation.

Abstract PO-082: Automated tumor segmentation, grading, and analysis of tumor heterogeneity in preclinical models of lung adenocarcinoma

Abstract Preclinical mouse models of lung adenocarcinoma are invaluable for the discovery of molecular drivers of tumor formation, progression, and therapeutic resistance. Histological analyses of these preclinical models require significant investments of time and training to ensure accuracy and consistency. Analysis by a clinical pathologist is the gold standard in this approach, but may be difficult to obtain due to the cost and availability of their services. As an alternative we have developed a digital pathology tool to identify, segment, grade, and analyze tumors in mouse models of lung adenocarcinoma. This convolutional neural network (CNN) model, based on ResNet18, was trained to classify normal lung tissue, normal airways, and the different grades (1 – 4) of lung adenocarcinoma from 100,000 224 × 224 pixel image patches (~16,000 patches per class). Our training dataset was constructed from whole slide images of hematoxylin and eosin stained lung sections from 4 different mouse models of lung adenocarcinoma with oncogenic Kras (KrasG12D/+), in combination with oncogenic p53 mutations (KrasG12D/+; p53R172H/+ and KrasG12D/+;p53R270H/+), or with the loss of the tumor suppressive TAp73 (KrasG12D/+;TAp7fltd/fltd). Our CNN demonstrated a strong correspondence with human pathologists on our holdout dataset, achieving a micro-F1 score of 0.81 on a pixel-by-pixel basis. As a test of our CNN, we analyzed two mouse models to better understand the role of TAp73 in lung adenocarcinoma: KrasG12D/+ (“K”) and KrasG12D/+;TAp73fltd/fltd (“TK”). Both human raters and our CNN reported a significant increase in the tumor burden of the compound mutant “TK” mice compared to the single mutant “K” mice. According to our CNN, this increased tumor burden was driven primarily by an increase in tumor size and not an increased number of tumors in “TK” mice. Because our CNN can assign different grades to regions within the same image patch and tumor, we also uncovered a high degree of intratumor heterogeneity that was not reported by the human pathologists, who are trained to assign one grade to a single tumor with a bias for the highest grade present in a given tumor. The finer grading resolution allowed our CNN to uncover the increased tumor size observed in the “TK” mice was due to expansion of Grade 2 regions (characterized by enlarged nuclei without irregular shape) within tumors that would be considered a higher grade by pathologists. Our CNN also provides a detailed map of tumor grades overlaid on the H&E images used for analysis, allowing for precise targeting of regions within tumors with other assays. We are currently utilizing these outputs in conjunction with other assays, such as spatial transcriptomic analysis and immunohistochemistry, to investigate the molecular mechanisms that underlie the expansion of Grade 2 tumor regions in “TK” mice. Future work will expand this tool into a multidimensional digital pathology pipeline that can accelerate current investigations and reveal new therapeutic targets and prognostic markers. Citation Format: John H. Lockhart, Hayley D. Ackerman, Kyubum Lee, Mahmoud Abdalah, Andrew Davis, Nicole Montey, Theresa Boyle, James Saller, Ayensur Keske, Kay Hänggi, Brian Ruffell, Olya Stringfield, Aik Choon Tan, Elsa R. Flores. Automated tumor segmentation, grading, and analysis of tumor heterogeneity in preclinical models of lung adenocarcinoma [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr PO-082.

Editorial Computational Pathology

Until recent years, histopathologists have analyzed tissue sections and diagnosed diseases including cancer primarily by using a microscope. The introduction of high resolution and high throughput digital scanners has enabled digitizing entire glass slides to generate high-resolution whole-slide images (WSI), de facto giving rise to the field of Digital Pathology. Since then, an increasing number of pathology laboratories have transitioned to a digital pipeline, which offers advantages such as remote diagnosis (e.g., for a second opinion), reduction of some routine work in the lab and partly reduction of physical storage. However, perhaps the most revolutionizing aspect of digital pathology is that it enables image analysis in pathology using machine learning. This field has come to be known as Computational Pathology.

Exchanging image processing and OCR components in a Setswana digitisation pipeline

As more natural language processing (NLP) applications benefit from neural network based approaches, it makes sense to re-evaluate existing work in NLP. A complete pipeline for digitisation includes several components handling the material in sequence. Image processing after scanning the document has been shown to be an important factor in final quality. Here we compare two different approaches for visually enhancing documents before Optical Character Recognition (OCR), (1) a combination of ImageMagick and Unpaper and (2) OCRopus. We also compare Calamari, a new line-based OCR package using neural networks, with the well-known Tesseract 3 as the OCR component. Our evaluation on a set of Setswana documents reveals that the combination of ImageMagick/Unpaper and Calamari improves on a current baseline based on Tesseract 3 and ImageMagick/Unpaper with over 30%, achieving a mean character error rate of 1.69 across all combined test data.

Abstract PO-023: Spatial genomics coupled with machine learning to identify p53-driven molecular signatures that are predictive of lung adenocarcinoma progression

Abstract P53 is frequently mutated in a wide variety of tumors; yet, the regulation and expression of downstream targets during tumor progression and response to therapy is unknown. Here, we use Kras/p53-driven lung adenocarcinoma in the mouse as a model system to identify p53-driven molecular signatures that predict lung adenocarcinoma progression. Using these tumors, we have developed a digital pathology tool using machine learning to grade lung adenocarcinomas. To do this, we analyzed hematoxylin and eosin (H&E) from mouse models of lung adenocarcinoma with Kras (KrasG12D/+) and in combination with p53 mutations (KrasG12D/+; p53R172H/+ and KrasG12D/+; p53R270H/+) and loss of TAp73 (KrasG12D/+; TAp73Δtd/Δtd). After grading, slides were divided into approximately 100,000 patches with dimensions of 224 × 224 pixels (113 × 113 µm). A convolutional neural network (CNN) model based on ResNet18 was trained to classify normal lung tissue, normal airways, and the different grades (1–4) of lung adenocarcinoma using 16,000 patches of each class. The resulting classification maps were used for analyses of tumor burden and progression. Adjacent tissue sections used for immunohistochemistry were co-registered to mapped H&E sections to investigate correlation between protein expression and tumor grade. Our CNN demonstrated a strong correspondence with human pathologists on our holdout dataset (81% agreement). Because our CNN can assign different grades to different regions within an image patch, we also uncovered a high degree of intratumor heterogeneity that was missed by human pathologists who assigned grades to entire tumors in a homogeneous manner. Both the pathologists and the CNN reported a significant increase in the tumor burden of compound mutant mice (KrasG12D/+; p53R172H/+, KrasG12D/+; p53R270H/+, and KrasG12D/+; TAp73Δtd/Δtd) compared to KrasG12D/+ mice. In addition, compound mutant mice were noted to have a greater proportion of high-grade (3-4) tumors by both approaches. In conclusion, our CNN demonstrates a high degree of agreement with human pathologists. Furthermore, this computational approach drastically increases the resolution of tumor grading in our mouse models and can detect regions of different grades within a single tumor. We are currently using single cell transcriptome analysis to define p53-molecular signatures that predict lung adenocarcinoma progression and grade. Future work will expand this tool into a multidimensional digital pathology pipeline that can accelerate current investigations and reveal new therapeutic targets and prognostic markers. Citation Format: John H. Lockhart, Kyubum Lee, Hayley D. Ackerman, Mahmoud Abdulah, Andrew Davis, Nicole Montey, Theresa Boyle, James Saller, Aysenur Keske, Kay Hanggi, Olya Stringfield, Aik Choon Tan, Elsa R. Flores. Spatial genomics coupled with machine learning to identify p53-driven molecular signatures that are predictive of lung adenocarcinoma progression [abstract]. In: Proceedings of the AACR Virtual Special Conference on Tumor Heterogeneity: From Single Cells to Clinical Impact; 2020 Sep 17-18. Philadelphia (PA): AACR; Cancer Res 2020;80(21 Suppl):Abstract nr PO-023.