Grid Of Tiles Research Articles

Abstract 7395: Histology-based prognosis prediction using deep learning outperforms and is independent of the MGMT methylation status in patients with glioblastoma

Abstract Introduction: MGMT gene promoter methylation status is today the only biomarker of survival for GBM patients and predicts favorable response to temozolomide (TMZ). Here we propose a deep learning-based model using histology data (H&E slides) to predict the prognosis of patients with Glioblastoma (GBM) treated by standard of care (SOC). Methods: We included TCGA GBM patients and selected only patients treated after 2005 (Stupp protocol start); compliant to WHO 2021 classification; treated by surgery and adjuvant therapy; with a follow-up superior to median overall survival (OS) (14.5 months). Short and long survivor labels were defined based on median OS cut-off. A deep learning pipeline was trained to predict these labels from the H&E slides (baseline surgery samples). First, the slides were automatically segmented into tissue and background, divided into a regular grid of tiles (112 × 112µm), and tiles were vectorized using a vision transformer-based feature extractor trained on H&E slides of 5,500 patients with 16 cancer subtypes (GBM excluded). In the second step, ABMIL deep learning model was trained to predict the survival category using the bag of tile features available from step 1 for each patient. The cohort was divided into five folds with even numbers of short and long survivors. For each fold, 50 ABMIL models were trained on the remaining folds and ensembled to predict the category of the patients in the fold. Training hyperparameters were tuned on a separated cohort excluding GBM patients. Results: After filtering, we analyzed n = 192 TCGA GBM patients including 154 patients with a known MGMT promoter methylation status. Our histology-based model had an accuracy of 0.661 and a ROC AUC of 0.706. It stratified the population into short and long survivors with an Hazard Ratio (HR) of 0.48 (95% CI 0.34-0.69). Our model outperformed the MGMT methylation status that had an HR of 0.57 (95% CI 0.40-0.83). Moreover, our model was independent of MGMT methylation status and the combination of both was able to further improve prognosis prediction: we applied our model independently to the 91 patients with an unmethylated MGMT status (predicted: 43 low, 48 high) and to the 63 patients with a methylated MGMT status (predicted: 30 low, 33 high), and obtained HR of 0.42 (95% CI 0.27-0.67) and 0.49 (95% CI 0.28-0.86) respectively. Conclusion: Having selected patients treated by the latest SOC and classified as GBM by the latest WHO criteria, we propose a model predicting survival based on histological specimens available in routine care, which outperformed the current unique biomarker MGMT methylation status. The combination of this status and our model improves prognosis predictions compared to each strategy alone. Citation Format: Lucas Fidon, Celine Thiriez, Alexandre Grimaldi, Omar Darwiche Domingues, Charles Maussion, Caroline Hoffmann. Histology-based prognosis prediction using deep learning outperforms and is independent of the MGMT methylation status in patients with glioblastoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 7395.

A Heterogeneous and Programmable Compute-In-Memory Accelerator Architecture for Analog-AI Using Dense 2-D Mesh

We introduce a highly heterogeneous and programmable compute-in-memory (CIM) accelerator architecture for deep neural network (DNN) inference. This architecture combines spatially distributed CIM memory array “tiles” for weight-stationary, energy-efficient multiply–accumulate (MAC) operations, together with heterogeneous special-function compute cores for auxiliary digital computation. Massively parallel vectors of neuron activation data are exchanged over short distances using a dense and efficient circuit-switched 2-D mesh, offering full end-to-end support for a wide range of DNN workloads, including CNNs, long-short-term-memory (LSTM), and transformers. We discuss the design of the “analog fabric”—the 2-D grid of tiles and compute cores interconnected by the 2-D mesh—and address the efficiency in both mapping of DNNs onto the hardware and in pipelining of various DNN workloads across a range of batch sizes. We show, for the first time, system-level assessments using projected component parameters for a realistic “analog AI” system, based on dense crossbar arrays of low-power nonvolatile analog memory elements, while incorporating a single common analog fabric design that can scale to large networks by introducing data transport between multiple analog AI chips. Our performance estimates for several networks, including large LSTM and bidirectional encoder representations from transformers (BERT), show highly competitive throughput while offering <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$40\times $ </tex-math></inline-formula> – <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$140\times $ </tex-math></inline-formula> higher energy efficiency than NVIDIA A100—thus illustrating the strong promise of analog AI and the proposed architecture for DNN inference applications.

Multi-scale segmentation algorithm for pattern-based partitioning of large categorical rasters

Analyzing large Earth Observation (EO) data on the broad spatial scales frequently involves regionalization of patterns. To automate this process we present a segmentation algorithm designed specifically to delineate segments containing quasi-stationary patterns. The algorithm is designed to work with patterns of a categorical variable. This makes it possible to analyze very large spatial datasets (for example, a global land cover) in their entirety. An input categorical raster is first tessellated into small square tiles to form a new, coarser, grid of tiles. A mosaic of categories within each tile forms a local pattern, and the segmentation algorithm partitions the grid of tiles while maintaining the cohesion of pattern in each segment. The algorithm is based on the principle of seeded region growing (SRG) but it also includes segment merging and other enhancements to segmentation quality. Our key contribution is an extension of the concept of segmentation to grids in which each cell has a non-negligible size and contains a complex data structure (histograms of pattern features). Specific modification of a standard SRG algorithm include: working in a distance space with complex data objects, introducing six-connected “brick wall” topology of the grid to decrease artifacts associated with tessellation of geographical space, constructing the SRG priority queue of seeds on the basis of local homogeneity of patterns, and using a content-dependent value of segment-growing threshold. The detailed description of the algorithm is given followed by an assessment of its performance on test datasets representing three pertinent themes of land cover, topography, and a high-resolution image. Pattern-based segmentation algorithm will find application in ecology, forestry, geomorphology, land management, and agriculture. The algorithm is implemented as a module of GeoPAT – an already existing, open source toolbox for performing pattern-based analysis of categorical rasters.

Multilabel Conditional Random Field Classification for UAV Images

In this letter, we formulate the multilabeling classification problem of unmanned aerial vehicle (UAV) imagery within a conditional random field (CRF) framework with the aim of exploiting simultaneously spatial contextual information and cross-correlation between labels. The pipeline of the framework consists of two main phases. First, the considered input UAV image is subdivided into a grid of tiles, which are processed thanks to an opportune representation and a multilayer perceptron classifier providing thus tile-wise multilabel prediction probabilities. In the second phase, a multilabel CRF model is applied to integrate spatial correlation between adjacent tiles and the correlation between labels within the same tile, with the objective to improve iteratively the multilabel classification map associated with the considered input UAV image. Experimental results achieved on two different UAV image data sets are reported and discussed.

Tile-Based Semisupervised Classification of Large-Scale VHR Remote Sensing Images

This paper deals with the problem of the classification of large-scale very high-resolution (VHR) remote sensing (RS) images in a semisupervised scenario, where we have a limited training set (less than ten training samples per class). Typical pixel-based classification methods are unfeasible for large-scale VHR images. Thus, as a practical and efficient solution, we propose to subdivide the large image into a grid of tiles and then classify the tiles instead of classifying pixels. Our proposed method uses the power of a pretrained convolutional neural network (CNN) to first extract descriptive features from each tile. Next, a neural network classifier (composed of 2 fully connected layers) is trained in a semisupervised fashion and used to classify all remaining tiles in the image. This basically presents a coarse classification of the image, which is sufficient for many RS application. The second contribution deals with the employment of the semisupervised learning to improve the classification accuracy. We present a novel semisupervised approach which exploits both the spectral and spatial relationships embedded in the remaining unlabelled tiles. In particular, we embed a spectral graph Laplacian in the hidden layer of the neural network. In addition, we apply regularization of the output labels using a spatial graph Laplacian and the random Walker algorithm. Experimental results obtained by testing the method on two large-scale images acquired by the IKONOS2 sensor reveal promising capabilities of this method in terms of classification accuracy even with less than ten training samples per class.

A Deep Learning Approach to UAV Image Multilabeling

In this letter, we face the problem of multilabeling unmanned aerial vehicle (UAV) imagery, typically characterized by a high level of information content, by proposing a novel method based on convolutional neural networks. These are exploited as a means to yield a powerful description of the query image, which is analyzed after subdividing it into a grid of tiles. The multilabel classification task of each tile is performed by the combination of a radial basis function neural network and a multilabeling layer (ML) composed of customized thresholding operations. Experiments conducted on two different UAV image data sets demonstrate the promising capability of the proposed method compared to the state of the art, at the expense of a higher but still contained computation time.

Texture Stationarization: Turning Photos into Tileable Textures

Texture synthesis has grown into a mature field in computer graphics, allowing the synthesis of naturalistic textures and images from photographic exemplars. Surprisingly little work, however, has been dedicated to synthesizing tileable textures, that is, textures that when laid out in a regular grid of tiles form a homogeneous appearance suitable for use in memory-sensitive real-time graphics applications. One of the key challenges in doing so is that most natural input exemplars exhibit uneven spatial variations that, when tiled, show as repetitive patterns. We propose an approach to synthesize tileable textures while enforcing stationarity properties that effectively mask repetitions while maintaining the unique characteristics of the exemplar. We explore a number of alternative measures for texture stationarity and show how each measure can be integrated into a standard texture synthesis method PatchMatch to enforce stationarity at user-controlled scales. We demonstrate the efficacy of our approach using a database of 118 exemplar images, both from publicly available sources as well as new ones captured under uncontrolled conditions, and we quantitatively analyze alternative stationarity measures for their robustness across many test runs using different random seeds. In conclusion, we suggest a novel synthesis approach that employs local histogram matching to reliably turn input photographs of natural surfaces into tiles well suited for artifact-free tiling.

The CrossSong Puzzle: Developing a Logic Puzzle for Musical Thinking

There is considerable interest in music-based games and apps. However, in existing games, music generally serves as an accompaniment or as a reward for progress. We set out to design a game where paying attention to the music would be essential to making deductions and solving the puzzle. The result is the CrossSong Puzzle, a novel type of music-based logic puzzle that integrates musical and logical reasoning. The game presents a player with a grid of tiles, each representing a mash-up of excerpts from two different songs. The goal is to rearrange the tiles so that each row and column plays a continuous musical excerpt. To create puzzles, we implemented an algorithm to automatically identify a set of song fragments to fill a grid such that each tile contains an acceptable mash-up. We present several optimisations to speed up the search for high-quality grids. We also discuss the iterative design of the game’s interface and present the results of a user evaluation of the final design. Finally, we present some insights learned from the experience which we believe are important to developing music-based puzzle games that are entertaining, feasible and that challenge one’s ability to think about music.

Voxel Level Radiologic-Pathologic Validation of Restriction Spectrum Imaging Cellularity Index with Gleason Grade in Prostate Cancer.

Restriction spectrum imaging (RSI-MRI), an advanced diffusion imaging technique, can potentially circumvent current limitations in tumor conspicuity, in vivo characterization, and location demonstrated by multiparametric magnetic resonance imaging (MP-MRI) techniques in prostate cancer detection. Prior reports show that the quantitative signal derived from RSI-MRI, the cellularity index, is associated with aggressive prostate cancer as measured by Gleason grade (GG). We evaluated the reliability of RSI-MRI to predict variance with GG at the voxel-level within clinically demarcated prostate cancer regions. Ten cases were processed using whole mount sectioning after radical prostatectomy. Regions of tumor were identified by an uropathologist. Stained prostate sections were scanned at high resolution (75 μm/pixel). A grid of tiles corresponding to voxel dimensions was graded using the GG system. RSI-MRI cellularity index was calculated from presurgical prostate MR scans and presented as normalized z-score maps. In total, 2,795 tiles were analyzed and compared with RSI-MRI cellularity. RSI-MRI cellularity index was found to distinguish between prostate cancer and benign tumor (t = 25.48, P < 0.00001). Significant differences were also found between benign tissue and prostate cancer classified as low-grade (GG = 3; t = 11.56, P < 0.001) or high-grade (GG ≥ 4; t = 24.03, P < 0.001). Furthermore, RSI-MRI differentiated between low and high-grade prostate cancer (t = 3.23; P = 0.003). Building on our previous findings of correlation between GG and the RSI-MRI among whole tumors, our current study reveals a similar correlation at voxel resolution within tumors. Because it can detect variations in tumor grade with voxel-level precision, RSI-MRI may become an option for planning targeted procedures where identifying the area with the most aggressive disease is important. Clin Cancer Res; 22(11); 2668-74. ©2016 AACR.

Multiclass Coarse Analysis for UAV Imagery

This paper presents a novel method to “coarsely” describe extremely high-resolution (EHR) images acquired by means of unmanned aerial vehicles (UAVs) over urban areas. Standard image analysis approaches cannot be directly exploited for the automatic description of UAV images due to their EHR. For this reason, we propose an alternative approach that consists first in the subdivision of the original UAV image in a grid of tiles. Then, each tile is compared with a library of training tiles to inherit the binary multilabel vector of the most similar training tile. This vector conveys a list of classes likely present in the considered tile. Our multiclass tile-based approach needs the definition of two main ingredients: 1) a suitable tile-representation strategy; and 2) a tile-to-tile matching operation. Various tile-representation and matching strategies are investigated. In particular, we present three global representation strategies, which process each tile as a whole and two point-based strategies that exploit points of interest within the considered tile. Regarding the matching strategies, two simple measures of distance, namely, the Euclidean and the chi-squared histogram distances, are explored. Interesting experimental results conducted on a rich set of real UAV images acquired over an urban area are reported and discussed.

PLUG

New protocols for the data link and network layer are being proposed to address limitations of current protocols in terms of scalability, security, and manageability. High-speed routers and switches that implement these protocols traditionally perform packet processing using ASICs which offer high speed, low chip area, and low power. But with inflexible custom hardware, the deployment of new protocols could happen only through equipment upgrades. While newer routers use more flexible network processors for data plane processing, due to power and area constraints lookups in forwarding tables are done with custom lookup modules. Thus most of the proposed protocols can only be deployed with equipment upgrades. To speed up the deployment of new protocols, we propose a flexible lookup module, PLUG (Pipelined Lookup Grid). We can achieve generality without loosing efficiency because various custom lookup modules have the same fundamental features we retain: area dominated by memories, simple processing, and strict access patterns defined by the data structure. We implemented IPv4, Ethernet, Ethane, and SEATTLE in our dataflow-based programming model for the PLUG and mapped them to the PLUG hardware which consists of a grid of tiles. Throughput, area, power, and latency of PLUGs are close to those of specialized lookup modules.

Histogram‐Based Approach for Automated Pavement‐Crack Sensing

This paper presents work towards the use of histogram‐based machine vision for the automated sensing of unprepared cracks in highway pavement. Machine vision is being developed for purposes of identifying and locating cracks in the size range from 1/8 to 1 in. (3–25 mm) in width in both asphalt concrete (AC) and portland cement concrete (PCC) pavements. The machine vision being developed is intended to work in real time in an automated crack‐sealing machine that will operate at approximately 2 mph. The method is based on segmenting the pavement image into a grid of tiles, and then representing each tile's properties by histogram‐based parameters. Comparison to a localized set of tiles provides information of crack existence and direction. This paper presents preliminary results showing the capability of the approach to find unprepared cracks in real pavements, and additionally, remaining work that will lead to its real‐time implementation is discussed.

Albedo maps of Pluto and Charon: Initial mutual event results

We present single-scattering albedo maps of the surfaces of Pluto and Charon based primarily on mutual event observations. The dataset contains 3374 photometric observations that cover 15 different satellite transit events, 14 satellite eclipse events, and other out-of-eclipse photometry spanning 1954 to 1986. The maps consist of a 59 × 29 grid of tiles for each body. We applied the technique of maximum entropy image reconstruction to invert the lightcurves, thus revealing surface maps of single-scattering albedo. The surface of Pluto is seen to have albedo features similar to our previous spot model maps (Buie and Tholen 1989). In particular, a south polar cap is evident in the map of Pluto. The north polar region is brighter than the equatorial regions but is not as bright as the south pole. Single-scattering albedos range from 0.98 in the south polar cap to a low near 0.2 at longitudes corresponding to the lightcurve minimum. The map of Charon is somewhat darker with single-scattering albedos as low as 0.03.