Complex Spatial Relationships Research Articles

A semantic guidance-based fusion network for multi-label image classification

Multi-label image classification (MLIC), a fundamental task assigning multiple labels to each image, has been seen notable progress in recent years. Considering simultaneous appearances of objects in the physical world, modeling object correlations is crucial for enhancing classification accuracy. This involves accounting for spatial image feature correlation and label semantic correlation. However, existing methods struggle to establish these correlations due to complex spatial location and label semantic relationships. On the other hand, regarding the fusion of image feature relevance and label semantic relevance, existing methods typically learn a semantic representation in the final CNN layer to combine spatial and label semantic correlations. However, different CNN layers capture features at diverse scales and possess distinct discriminative abilities. To address these issues, in this paper we introduce the Semantic Guidance-Based Fusion Network (SGFN) for MLIC. To model spatial image feature correlation, we leverage the advanced TResNet architecture as the backbone network and employ the Feature Aggregation Module for capturing global spatial correlation. For label semantic correlation, we establish both local and global semantic correlation. We further enrich model features by learning semantic representations across multiple convolutional layers. Our method outperforms current state-of-the-art techniques on PASCAL VOC (2007, 2012) and MS-COCO datasets.

Skip-patching spatial-temporal discrepancy-based anomaly detection on multivariate time series

Anomaly detection in the Industrial Internet of Things (IIoT) is a challenging task that relies heavily on the efficient learning of multivariate time series representations. We introduce Skip-patching and Spatial-Temporal discrepancy mechanisms to improve the efficiency of detecting anomalies. Traditional feature extraction is hindered by redundant information in limited datasets. The situation is that feature generation from stable operational processes results in low-quality representations. To address this challenge, we propose the Skip-Patching mechanism. This approach involves selectively extracting features from partial data patches, prompting the model to learn more meaningful knowledge through self-supervised learning. It also effectively doubles the training sample size by creating independent sub-groups of patches. Despite the complex spatial and temporal relationships in IIoT systems, existing methods mainly extracted features from a single domain, either temporal or spatial (sensor-wise), or simply cascaded two features, i.e., one after one, which limited anomaly detection capabilities. To address this, we introduce the Spatial-Temporal Association Discrepancy component, which leverages discrepancies between spatial and temporal features to enhance latent representation learning. Our Skip-Patching Spatial-Temporal Anomaly Detection (SSAD) framework combines these two components to provide a more diverse and comprehensive learning process. Tested across four multivariate time series anomaly detection benchmarks, SSAD demonstrates superior performance, confirming the efficacy of combining Skip-patching and Spatial-Temporal features to enhance anomaly detection in IIoT systems.

Millimeter wave–3D massive MIMO: Deep prior‐aided graph neural network combining with hierarchical residual learning for beamspace channel estimation

SummaryMillimeter Wave (mmWave) communication has emerged as a transformative technology at the forefront of wireless communication. One of the key challenges in harnessing the potential of mmWave technology is overcoming the increased susceptibility to propagation losses and environmental obstacles. To address these challenges, Three‐Dimensional Massive Multiple‐Input Multiple‐Output (3D Massive MIMO) systems have gained traction. The 3D aspect extends this concept by considering the elevation dimension, allowing for enhanced spatial resolution and coverage. Accurate estimation of the channel in 3D Massive MIMO scenarios is particularly challenging because of the complex propagation characteristics of mmWave signals. This paper introduces an efficient‐Aided Graph Neural Network Combining with Hierarchical Residual Learning (DPrGNN‐HrResNetL), designed specifically for beamspace Channel Estimation (CE)in mmWave‐Massive MIMO environments. The proposed model leverages deep priors and GNN mechanisms to enhance the extraction of spatial features, while hierarchical residual connections facilitate effective information flow through the network. DPrGNN enables the model to capture and understand complex spatial relationships among different antenna elements. The incorporation of deep priors provides a mechanism for leveraging prior knowledge about channel characteristics. This enhances the efficiency of the learning process, allowing the model to learn and adapt more effectively. The integration of hierarchical residual connections facilitates effective information flow through the network. This is particularly important for modeling complex dependencies within the beamspace channel data, enhancing the learning capacity of the network. The performance of the DPrGNN‐HrResNetL model is evaluated across a range of Signal‐to‐Noise Ratios (SNRs), utilizing metrics such as Normalized Mean Squared Error (NMSE) to measure the accuracy of the estimation. The outcomes underscore the resilience and efficacy of the DPrGNN‐HrResNetL approach in achieving precise CE within demanding mmWave scenarios.

Prediction of dose distributions for non-small cell lung cancer patients using MHA-ResUNet.

The current level of automation in the production of radiotherapy plans for lung cancer patients is relatively low. With the development of artificial intelligence, it has become a reality to use neural networks to predict dose distributions and provide assistance for radiation therapy planning. However, due to the significant individual variability in the distribution of non-small cell lung cancer (NSCLC) planning target volume (PTV) and the complex spatial relationships between the PTV and organs at risk (OARs), there is still a lack of a high-precision dose prediction network tailored to the characteristics of NSCLC. To assist in the development of volumetric modulated arc therapy (VMAT) plans for non-small cell lung cancer patients, a deep neural network is proposed to predict high-precision dose distribution. This study has developed a network called MHA-ResUNet, which combines a large-kernel dilated convolution module and multi-head attention (MHA) modules. The network was trained based on 80 VMAT plans of NSCLC patients. CT images, PTV, and OARs were fed into the independent input channel. The dose distribution was taken as the output to train the model. The performance of this network was compared with that of several commonly used networks, and the networks' performance was evaluated based on the voxel-level mean absolute error (MAE) within the PTV and OARs, as well as the error in clinical dose-volume metrics. The MAE between the predicted dose distribution and the manually planned dose distribution within the PTV is 1.43Gy, and the D95 error is less than 1Gy. Compared with the other three commonly used networks, the dose error of the MHA-ResUNet is the smallest in PTV and OARs. The proposed MHA-ResUNet network improves the receptive field and filters the shallow features to learn the relative spatial relation between the PTV and the OARs, enabling accurate prediction of dose distributions in NSCLC patients undergoing VMAT radiotherapy.

Representative co-location pattern post-mining based on maximal row instances representation model

The mining result set of spatial prevalent co-location patterns(SPCPs) is often large and redundant, especially when the prevalence threshold is set to low or long SPCPs are present. Meanwhile, the distribution of SPCPs in continuous space and complex spatial relationships with each other of spatial data make the compression and reorganization of SPCPs a challenging problem. To solve this problem, in this paper, a representative co-location pattern mining framework based on the maximal row instance(MRI) representation model is proposed. First, the MRI representation model is proposed to effectively preserve the pattern distribution information of prevalent co-location patterns. To establish the MRI representation model, the basic algorithm and geometric algorithm are proposed in this paper. Two materialization methods based on the MRI representation model, 0-1 vector and key–value vector, are presented. Secondly, the similarity measure of SPCPs under the context of the MRI representation model is proposed, which calculates the similarity between any two co-location patterns without adding additional information such as domain background. Furthermore, the mining framework based on the k nearest neighbors density peak clustering algorithm is presented to extract representative co-location patterns. Finally, the efficiency and scalability of the proposed method are verified on the synthetic data and real data. Compared with the existing methods, the representative co-location patterns of the proposed method have well compression performances.

Modified osprey algorithm for optimizing capsule neural network in leukemia image recognition

The diagnosis of leukemia is a serious matter that requires immediate and accurate attention. This research presents a revolutionary method for diagnosing leukemia using a Capsule Neural Network (CapsNet) with an optimized design. CapsNet is a cutting-edge neural network that effectively captures complex features and spatial relationships within images. To improve the CapsNet's performance, a Modified Version of Osprey Optimization Algorithm (MOA) has been utilized. Thesuggested approach has been tested on the ALL-IDB database, a widely recognized dataset for leukemia image classification. Comparative analysis with various machine learning techniques, including Combined combine MobilenetV2 and ResNet18 (MBV2/Res) network, Depth-wise convolution model, a hybrid model that combines a genetic algorithm with ResNet-50V2 (ResNet/GA), and SVM/JAYA demonstrated the superiority of our method in different terms. As a result, the proposed method is a robust and powerful tool for diagnosing leukemia from medical images.

Computer-aided diagnosis of Alzheimer’s disease and neurocognitive disorders with multimodal Bi-Vision Transformer (BiViT)

Cognitive disorders affect various cognitive functions that can have a substantial impact on individual’s daily life. Alzheimer’s disease (AD) is one of such well-known cognitive disorders. Early detection and treatment of cognitive diseases using artificial intelligence can help contain them. However, the complex spatial relationships and long-range dependencies found in medical imaging data present challenges in achieving the objective. Moreover, for a few years, the application of transformers in imaging has emerged as a promising area of research. A reason can be transformer’s impressive capabilities of tackling spatial relationships and long-range dependency challenges in two ways, i.e., (1) using their self-attention mechanism to generate comprehensive features, and (2) capture complex patterns by incorporating global context and long-range dependencies. In this work, a Bi-Vision Transformer (BiViT) architecture is proposed for classifying different stages of AD, and multiple types of cognitive disorders from 2-dimensional MRI imaging data. More specifically, the transformer is composed of two novel modules, namely Mutual Latent Fusion (MLF) and Parallel Coupled Encoding Strategy (PCES), for effective feature learning. Two different datasets have been used to evaluate the performance of proposed BiViT-based architecture. The first dataset contain several classes such as mild or moderate demented stages of the AD. The other dataset is composed of samples from patients with AD and different cognitive disorders such as mild, early, or moderate impairments. For comprehensive comparison, a multiple transfer learning algorithm and a deep autoencoder have been each trained on both datasets. The results show that the proposed BiViT-based model achieves an accuracy of 96.38% on the AD dataset. However, when applied to cognitive disease data, the accuracy slightly decreases below 96% which can be resulted due to smaller amount of data and imbalance in data distribution. Nevertheless, given the results, it can be hypothesized that the proposed algorithm can perform better if the imbalanced distribution and limited availability problems in data can be addressed.Graphical abstract

EUNet: Edge-UNet for Accurate Building Extraction and Edge Emphasis in Gaofen-7 Images

Deep learning is currently the mainstream approach for building extraction tasks in remote-sensing imagery, capable of automatically learning features of buildings in imagery and yielding satisfactory extraction results. However, due to the diverse sizes, irregular layouts, and complex spatial relationships of buildings, extracted buildings often suffer from incompleteness and boundary issues. Gaofen-7 (GF-7), as a high-resolution stereo mapping satellite, provides well-rectified images from its rear-view imagery, which helps mitigate occlusions in highly varied terrain, thereby offering rich information for building extraction. To improve the integrity of the edges of the building extraction results, this paper proposes a dual-task network (Edge-UNet, EUnet) based on UNet, incorporating an edge extraction branch to emphasize edge information while predicting building targets. We evaluate this method using a self-made GF-7 Building Dataset, the Wuhan University (WHU) Building Dataset, and the Massachusetts Buildings Dataset. Comparative analysis with other mainstream semantic segmentation networks reveals significantly higher F1 scores for the extraction results of our method. Our method exhibits superior completeness and accuracy in building edge extraction compared to unmodified algorithms, demonstrating robust performance.

TransLevelSet: Integrating vision transformers with level-sets for medical image segmentation

Recently, Vision Transformers (ViTs) have emerged as a breakthrough in computer vision and image analysis. Still, their exceptional performance depends on the availability of large amounts of annotated training data and the availability of considerable computational resources. This naturally raises the risk of overfitting and limited generalization capability in settings with limited training data. Aiming to cope with this issue, we propose the Level-Set Transformer (TransLevelSet), a hybrid methodology that encompasses an additional term in the loss function used in the ViT learning process, which has been originally defined in the context of Level-Set (LS) deformable models. This loss term avails the spatial information obtained by the level-set energy terms for image segmentation and mitigates the dependency of ViTs on the amount of available data. Moreover, this level-set loss promotes smooth and topologically consistent demarcation of structures, taking advantage of the capacity of ViTs to capture complex spatial relationships and contextual information. The main contributions of this work include: a) a pioneering approach to the integration of ViTs with level-sets; and, b) the application of the proposed methodology on two case studies related to cancer, namely the malignant melanoma, and colon cancer. We evaluate TransLevelSet on three different publicly available benchmark datasets for medical image segmentation, that include dermoscopic and histopathological images. The results of the experiments demonstrate consistent gains in terms of generalization capability introduced by the LS terms.

3D exploration of gene expression in chicken embryos through combined RNA fluorescence in situ hybridization, immunofluorescence, and clearing

BackgroundFine characterization of gene expression patterns is crucial to understand many aspects of embryonic development. The chicken embryo is a well-established and valuable animal model for developmental biology. The period spanning from the third to sixth embryonic days (E3 to E6) is critical for many organ developments. Hybridization chain reaction RNA fluorescent in situ hybridization (HCR RNA-FISH) enables multiplex RNA detection in thick samples including embryos of various animal models. However, its use is limited by tissue opacity.ResultsWe optimized HCR RNA-FISH protocol to efficiently label RNAs in whole mount chicken embryos from E3.5 to E5.5 and adapted it to ethyl cinnamate (ECi) tissue clearing. We show that light sheet imaging of HCR RNA-FISH after ECi clearing allows RNA expression analysis within embryonic tissues with good sensitivity and spatial resolution. Finally, whole mount immunofluorescence can be performed after HCR RNA-FISH enabling as exemplified to assay complex spatial relationships between axons and their environment or to monitor GFP electroporated neurons.ConclusionsWe could extend the use of HCR RNA-FISH to older chick embryos by optimizing HCR RNA-FISH and combining it with tissue clearing and 3D imaging. The integration of immunostaining makes possible to combine gene expression with classical cell markers, to correlate expressions with morphological differentiation and to depict gene expressions in gain or loss of function contexts. Altogether, this combined procedure further extends the potential of HCR RNA-FISH technique for chicken embryology.

A CNN-Based Framework for Video Analysis and Accident Detection

This research investigates the development and deployment of a Convolutional Neural Network (CNN) model for automatic accident detection in CCTV footage. The ever-increasing reliance on video surveillance necessitates efficient and accurate methods for accident identification. CNNs, with their inherent ability to learn complex spatial relationships within images, are particularly well-suited for this task. This study proposes a CNN architecture that utilizes a pre-trained MobileNetV2 base for feature extraction, followed by a custom classification head tailored to the specific task of accident vs. no accident classification. The model is trained on a dataset of grayscale video frames, achieving an impressive accuracy of 92% on the testing set. This high level of accuracy suggests that CNNs hold significant promise for real-world accident detection applications. Furthermore, to bridge the gap between research and practical implementation, the model is converted to a TensorFlow Lite (TFLite) format for deployment on resource-constrained devices. Additionally, a user-friendly frontend application is developed, empowering users to interact with the model and analyze both images and videos. This user-centric approach broadens the model's accessibility and paves the way for potential improvements in road safety through real-time accident detection.

DSTN: Dynamic Spatio-Temporal Network for Early Fault Warning in Chemical Processes

Multivariate time series prediction, especially in early fault warning for chemical processes, poses significant challenges. The advent of graph neural network (GNN) method has made breakthroughs in this domain by enabling the processing of topological data. However, the traditional methods suffer from the issue of over-smoothing and inability to capture intricate multi-scale spatio-temporal dependencies. Additionally, the existing graph structures fall short in describing the complex spatial relationships among multi-stage sensors, impeding their adaptability to dynamically evolving chain reaction scenarios. To alleviate these limitations, a novel Dynamic spatio-Temporal Network for early fault warning in chemical processes, named DSTN for short, is proposed in this paper. We extract the spatial and temporal features of the time series by the designed dynamic GNN and the improved Transformer network. Then, we integrate the spatio-temporal features through the residual network. DSTN has the following advantages: (1) A one-dimensional convolutional neural network is seamlessly incorporated into the Transformer architecture for bolstering its capacity to discern both global and local features within time series. (2) The continuous sliding window and mutual information methods are employed to construct a dynamic topology graph, and a K-order adjacency matrix is designed to rectify the inefficiencies in learning weights associated with convolution kernel parameters. (3) Multiple spatio-temporal modules interconnected via residual connection to adaptively fuse multi-scale features. Experimental results demonstrate that our proposed DSTN method outperforms existing methods in terms of both performance and interpretability in early fault warning of chemical processes.

A combination network based on graph modules for cardiac segmentation

BackgroundCardiac segmentation plays a crucial role in extracting cardiac substructures from computed tomography images, providing necessary information for subsequent cardiac disease diagnostics. Current research on cardiac segmentation algorithms faces challenges such as complex spatial distributions and limited labeled samples. The preferred method, convolutional neural network (CNN), excels at capturing features at various levels. However, the coarse convolutional operations of CNN often neglect valuable structural information. In contrast, graph neural network (GNN) excels at capturing complex spatial relationships among nodes by aggregating messages from graphs. MethodologyIn this work, a GNN-CNN combination network (GCCN) with enhanced feature extraction by GNN is proposed for cardiac segmentation. The combination network integrates the strengths of two approaches while compensating for each other's shortcomings. Firstly, 3D CT images are transformed into graphs, whose nodes with similar feature values within a certain spatial proximity are connected. Secondly, a lightweight graph attention network (LGAT) module is proposed for aggregating the messages of constructed graphs. Thirdly, the data, resampled from the feature-enhanced graphs, is fed into the 3D ResU-Net for the final segmentation process. ResultThe proposed method achieves an average Dice score of 92.98% which is 1.84% higher than 3D ResU-Net. The average Dice score of each cardiac substructure improved, particularly for the ascending aorta, achieving 96.35%, an 8.29% improvement over 3D ResU-Net. Additionally, our study demonstrates that GCCN has the improved adaptability in handling datasets with limited samples. ConclusionIn general, GCCN allows for the extraction of richer structural information from CT images, resulting in stronger feature representation of the data. Compared to pure convolutional methods, GCCN not only enhances segmentation accuracy but also exhibits greater adaptability to small-sample data. GCCN represents an exploration of applying graph neural networks to cardiac segmentation tasks from a novel perspective.

A Novel Underground Culvert Detection and Positioning System Projected along the Public Vertical Line

The limitations of existing procedures make it difficult to locate and identify old subterranean culverts in urban infrastructure management. In order to effectively manage urban infrastructure, subterranean pipe culverts must be accurately located and detected. In this research, we investigate the method of computing the shortest distance from the point to the ellipse and propose a pipeline collision detection method based on the projection of the direction of the common perpendicular. In the positioning accuracy test, we simulate the detection of straight and curved paths and obtain satisfactory results; the experimental results show that the detection errors are within acceptable limits for different azimuth and bending angles; in the correctness test, we compared with AutoCAD and ArcGIS, and found that the algorithm in this paper shows superiority in collision detection, especially when dealing with complex spatial relationships and large amounts of data, with evident efficiency advantages. Through theoretical analysis and experimental verification, we demonstrate the effectiveness and reliability of the method.

Abstract 2309: 3D spatial quantification of lymphocyte infiltration and collagen features in the tumor microenvironment using a novel assay: 3D I/O Pro™

Abstract Background: The composition of the tumor microenvironment (TME) is a major determinant of response to therapy in many solid tumors. To-date, characterization of the TME has been based on limited analysis of thin tissue sections. Here we demonstrate the utility of an end-to-end 3D spatial biology workflow, the 3D I/O Pro™, based on whole tissue imaging, to identify and quantify tumor cells, lymphocytes, and collagen features in human FFPE tissue samples. Methods: Five human colorectal cancer FFPE blocks were deparaffinized, stained with nuclear (TO-PRO-3) and general protein (eosin) dyes, and cleared using a modified iDISCO protocol. Entire samples were imaged at 2 µm/pixel resolution with a hybrid open-top light-sheet microscope, the 3Di™. Regions of interest (ROIs) with a volume of 0.5 mm3 were re-imaged at 0.33 µm/pixel. Cell nuclei and collagen were segmented and 3D spatial relationships between tumor cells, lymphocytes, and collagen fibers were quantified. Analyses were performed within select ROIs using 3Dai™ tools including U-Net, CytoMAP, CT-FIRE, CurveAlign, and custom Python scripts. Results: The ratio of stromal to tumor parenchymal lymphocytes (lymphocyte infiltration ratio) varied from 1.4 up to 9.1 in 3D volumes and 1 to 25 in 2D virtual sections taken throughout all 5 CRC samples. The ratio of the collagen fibers within 100 µm of the tumor border-oriented perpendicular to the tumor-stromal boundary compared to those oriented parallel to the tumor-stromal boundary was 0.37 in a proof-of-concept evaluation of a 2D virtual section with a lymphocyte infiltration ratio of 12. Qualitatively, areas with perpendicular collagen had more lymphocyte infiltration into the tumor parenchyma than areas with parallel collagen orientation. Conclusions: We demonstrated that the 3D I/O Pro™ pipeline can quantify lymphocyte density in tumor parenchyma and stroma and analyze collagen features, including orientation, within 3D ROIs in the TME. This workflow allows us to characterize tumors based on many complex spatial relationships and could have broad applicability in research and development of novel cancer therapies that target tumor fibrosis or other features of the TME. In the future, we plan to correlate features from the 3D TME with response to immunotherapy and use these features to refine histologic definitions of immune spatial phenotypes. Citation Format: Caleb Stoltzfus, Jasmine Wilson, Nathan Grant, Alexandra Alvarsson, Anna DeWitt, David Simmons, Brandy Olin, Bonnie Phillips, Laura A. Dillon, G. Travis Clifton, Nicholas P. Reder. 3D spatial quantification of lymphocyte infiltration and collagen features in the tumor microenvironment using a novel assay: 3D I/O Pro™ [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2309.

Abstract 7433: CGC-hosted comprehensive pipeline for spatial transcriptomics analysis

Abstract Spatial transcriptomics has been experiencing significant growth in the past years. Inspired by two widely adopted methodologies, particularly in situ hybridization and next-generation sequencing, with an emphasis on the single-cell RNA sequencing (scRNA-seq), this hybrid method facilitates whole transcriptome profiling while preserving spatial context at high resolutions, thus allowing the exploration of novel insights in cancer research. We introduce the highly configurable solution for sequencing-based technologies that allow comprehensive spatial analysis. The pipeline is available on NCI-funded Cancer Genomics Cloud (CGC) platform, powered by Seven Bridges. The CGC platform provides a collaborative cloud base computation infrastructure that collocates computation, over 1000 bioinformatics workflows, and 4+ PB data to researchers, making the analysis of the Cancer Research data Commons (CRDC) datasets accessible from any environment. This pipeline is developed using reliable, widely adopted packages and can process datasets generated by several leading technologies. It comprises a set of common steps including quality control, data preprocessing, dimensionality reduction, cluster identification, detection of spatially variable features, and integration with scRNA-Seq reference. The pipeline is highly configurable, allowing for various settings to be modified and optimized for result refinement. Additionally, it allows the selection of only specific components of the pipeline to be executed. The key steps of the pipeline are visually represented, allowing researchers to get a detailed insight into results. Here, we demonstrate a typical flow of spatial transcriptomics analysis on publicly available datasets using the developed pipeline. We illustrate the impact of pipeline settings on the analysis outcomes. Furthermore, we identify spatially variable genes whose expressions show a distinct localization within the tissue. Finally, we perform the data integration to predict the cell type composition within the determined spatial domains. Spatial transcriptomics analysis is a powerful method that has highly improved critical aspects of cancer research, such as the characterization of tumor microenvironment, the discovery of novel biomarkers and the clarification of drug resistance mechanisms. The ongoing evolution of this method is expected to play vital role in our deep understanding of complex spatial relationships within tissues. This CGC-hosted workflow is developed to contribute to the promising advancements. Citation Format: Miona Rankovic, Nevena Vukojicic, Nevena Ilic Raicevic, Vida Matovic, Divya Sain, Jack DiGiovanna, Brandi Davis-Dusenbery. CGC-hosted comprehensive pipeline for spatial transcriptomics analysis [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 7433.

Multi‐agent trajectory prediction with adaptive perception‐guided transformers

AbstractThe ability to predict the trajectory of an autonomous vehicle accurately is crucial for safe and efficient navigation. However, predicting diverse and multimodal futures can be challenging. Recent approaches such as attention and graph neural networks have achieved state‐of‐the‐art performance by considering agent interactions and map contexts. This study focused on multi‐agent prediction using an agent‐centric approach with transformers. This enables parallel computation and a comprehensive understanding of the environment. Two main features are introduced: an adaptive receptive field (ARF) that captures the relevant surroundings for each agent, and perception encoding, which serves as spatial context embeddings. The ARF adapts to the agent's velocity and rotation, focusing attention ahead at high speeds or to the sides when it is slower. Perception encoding divides agents or lanes into levels and encodes the information of each level. This approach enables the efficient encoding of complex spatial relationships. The proposed method combines these advances with transformer modelling for multi‐agent trajectory prediction while ensuring real‐time prediction capabilities. The approach is evaluated on the Argoverse benchmark and better performance than the state‐of‐the‐art baseline is achieved. By addressing challenges such as multimodal outputs and robustness, the study enhances the safety and efficiency of autonomous driving systems by more accurately predicting trajectories.

Generative approaches for solving tangram puzzles

The Tangram is a dissection puzzle composed of seven polygonal pieces that can form different patterns. Solving the Tangram is an irregular shape packing problem known to be NP-hard. This paper investigates the application of four deep-learning architectures—Convolutional Autoencoder, Variational Autoencoder, U-Net, and Generative Adversarial Network—specifically designed for solving Tangram puzzles. We explore the potential of these architectures in learning the complex spatial relationships inherent in Tangram configurations. Our experiments show that the Generative Adversarial Network competes well with other architectures and converges considerably faster. We further prove that traditional evaluation metrics based on pixel accuracy often fail in assessing the visual quality of the generated Tangram solutions. We introduce a loss function based on a Weighted Mean Absolute Error that prioritizes pixels representing inter-piece sections over those covered by individual pieces. Extending this loss function, we propose a novel evaluation metric as a more fitting measure for assessing Tangram solutions compared to traditional metrics. This investigation advances our understanding of the capabilities of artificial intelligence in complex geometrical problem domains.

Quantitative multiplex immunohistochemistry reveals inter-patient lymphovascular and immune heterogeneity in primary cutaneous melanoma.

Quantitative, multiplexed imaging is revealing complex spatial relationships between phenotypically diverse tumor infiltrating leukocyte populations and their prognostic implications. The underlying mechanisms and tissue structures that determine leukocyte distribution within and around tumor nests, however, remain poorly understood. While presumed players in metastatic dissemination, new preclinical data demonstrates that blood and lymphatic vessels (lymphovasculature) also dictate leukocyte trafficking within tumor microenvironments and thereby impact anti-tumor immunity. Here we interrogate these relationships in primary human cutaneous melanoma. We established a quantitative, multiplexed imaging platform to simultaneously detect immune infiltrates and tumor-associated vessels in formalin-fixed paraffin embedded patient samples. We performed a discovery, retrospective analysis of 28 treatment-naïve, primary cutaneous melanomas. Here we find that the lymphvasculature and immune infiltrate is heterogenous across patients in treatment naïve, primary melanoma. We categorized five lymphovascular subtypes that differ by functionality and morphology and mapped their localization in and around primary tumors. Interestingly, the localization of specific vessel subtypes, but not overall vessel density, significantly associated with the presence of lymphoid aggregates, regional progression, and intratumoral T cell infiltrates. We describe a quantitative platform to enable simultaneous lymphovascular and immune infiltrate analysis and map their spatial relationships in primary melanoma. Our data indicate that tumor-associated vessels exist in different states and that their localization may determine potential for metastasis or immune infiltration. This platform will support future efforts to map tumor-associated lymphovascular evolution across stage, assess its prognostic value, and stratify patients for adjuvant therapy.

Deep Machine Learning for Medical Diagnosis, Application to Lung Cancer Detection: A Review

Deep learning has emerged as a powerful tool for medical image analysis and diagnosis, demonstrating high performance on tasks such as cancer detection. This literature review synthesizes current research on deep learning techniques applied to lung cancer screening and diagnosis. This review summarizes the state-of-the-art in deep learning for lung cancer detection, highlighting key advances, limitations, and future directions. We prioritized studies utilizing major public datasets, such as LIDC, LUNA16, and JSRT, to provide a comprehensive overview of the field. We focus on deep learning architectures, including 2D and 3D convolutional neural networks (CNNs), dual-path networks, Natural Language Processing (NLP) and vision transformers (ViT). Across studies, deep learning models consistently outperformed traditional machine learning techniques in terms of accuracy, sensitivity, and specificity for lung cancer detection in CT scans. This is attributed to the ability of deep learning models to automatically learn discriminative features from medical images and model complex spatial relationships. However, several challenges remain to be addressed before deep learning models can be widely deployed in clinical practice. These include model dependence on training data, generalization across datasets, integration of clinical metadata, and model interpretability. Overall, deep learning demonstrates great potential for lung cancer detection and precision medicine. However, more research is required to rigorously validate models and address risks. This review provides key insights for both computer scientists and clinicians, summarizing progress and future directions for deep learning in medical image analysis.