Image Analysis Approach Research Articles

A bi-directional Long Short-Term Memory-based Diabetic Retinopathy detection model using retinal fundus images

Images of the retina are widely used for diagnosing fundus disease. Low-quality retinal photos make it hard for computer-aided diagnosis systems and ophthalmologists to make a clinical diagnosis. In ophthalmology, precision medicine is based partly on the quality of retinal images. Diabetic Retinopathy (DR) is a common complication of diabetes mellitus that causes iris damage. It is difficult to detect and, if not detected early, can result in blindness. Convolutional neural networks are gaining popularity as an effective deep learning (DL) approach for medical image analysis. This study suggests using deep learning approaches at various stages of the fundus image-based diagnostic pipeline for diabetic retinopathy (DR). Many fields, including medical image classification, have adopted DL representations. Using retinal fundus images, we propose a bi-directional extended short-term memory-based diabetic retinopathy detection model. By examining images of the retinal fundus, the Bi-directional Long Short-Term Memory (LSTM) method can detect and classify various grades of DR. As a preprocessing step, the proposed model uses the Multiscale Retinex with Chromaticity Preservation (MSRCP) method to increase the difference of fundus pictures and progress the short difference of medicinal views. To prepare satisfactory results for image processing, multiscale retinex with chromaticity preservation is used. However, choosing the parameters’ values, such as the Gaussian scales, gain, offset, etc., is the main difficulty with the retinex algorithm. To achieve a practical effect, these parameters must be adjusted. The main goal of the suggested method is to obtain the ideal values for the parameters used in the MSRCP algorithm. Also, photos that have already been processed are used to make feature vectors with the help of an efficient net-based feature extractor that uses deep learning. Many experiments use the benchmark Methods to Evaluate Segmentation and Indexing Techniques in the Field of Retinal Ophthalmology (MESSIDOR) dataset. The results are analyzed in terms of various evaluation factors. The results show that the Bi-LSTM-MSRCP technique is better at diagnosing DR than more modern methods.

Abstract 4620: Automated multiplex immunofluorescence workflow to interrogate the cellular composition of the tumor microenvironment

Abstract Background: The tumor microenvironment (TME) is a constantly changing niche due to dynamic interactions between tumor cells and their surroundings. Recently, it became evident that a better understanding of the TME is needed to target the disease more accurately (Binnewies et al., Nat Med 2018; Vitale et al., Nat Med 2021). Spatial hyperplex immunofluorescence enables the visualization of cellular and non-cellular tissue components simultaneously. However, the increasing number of biomarkers detected by spatial proteomics quickly escalates the complexity of images and renders their interpretation challenging. Extracting quantitative data from images remains a challenge as it requires extensive training and experience in data analysis. Additionally, inferring cellular interactions and dependencies from spatial assays is an important milestone that needs to be overcome with image analysis approaches. Here, we present an end-to-end solution that combines automated hyperplex execution with image data extraction to study the spatial composition of TME. Methods: Sequential hyperplex immunofluorescence was performed on COMET™ on an FFPE Breast Tissue Microarray (TMA), generating images containing 42 layers: nuclear DAPI, tissue autofluorescence, and 39 single layers for biomarkers. All layers were delivered as a single ome.tiff file and processed using HORIZON™ image analysis software in combination with downstream statistical analysis. The tissue composition was interrogated with the use of an in-house trained nuclei detection algorithm (Schmid et al., MICCAI 2018; Weigert et al., WACV 2020), in-house developed image artifact exclusion approach, and spatial analysis based on a Squidpy workflow (Palla G. et al., Nat Meth 2022). Results: We interrogated the composition of breast tumor tissues (n=9). Our in-house developed pipeline allows for filtering out both autofluorescent objects and segmentation artifacts based on the tissue autofluorescence and morphological features. Once the quality of segmented cell detection was assured, over 20 different cell phenotypes were identified by unsupervised Leiden clustering and visualized using UMAP dimensionality reduction method. It revealed several myeloid cell subsets highlighting their heterogeneous phenotypes including, but not limited to, immunosuppressive neutrophils and macrophages. Contrastingly, the uniform presence of T regulatory lymphocytes was detected in majority of cores and their co-occurrence with CD14+CD163+ macrophages was observed. Conclusion: Here, we showcased an innovative automated workflow that highlights the ease of adoption of multiplex imaging to explore TME composition at single-cell resolution using simple workflow: slide in-data out. This workflow is easily transferrable to various cohorts of specimens to provide a toolset for spatial cellular dissection of the cancer milieu. Citation Format: Vytautas Navikas, Quentin Juppet, Samuel Aubert, Benjamin Pelz, Joanna Kowal, Diego Dupouy. Automated multiplex immunofluorescence workflow to interrogate the cellular composition of the tumor microenvironment. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 4620.

Abstract 5362: AI-enabled novel workflow to evaluate T-cell activity in vitro using 3D spheroid models

Abstract T-cell therapies are designed to help our immune system eliminate cancer cells. Those include CAR T-cells (Chimeric Antigen Receptor engineered T-cells), tumor infiltrating lymphocytes (TIL), and other genetically modified T-cells. In recent years, the field of cell therapy has started to expand, including the launch of the first CAR T-cell therapies to treat blood cancer in 2017, which was a critical milestone in this field. Despite its boom, the discovery of novel immunotherapies that specifically enhance T-cell response against cancer cells remains a challenging task, limited by the absence of robust in vitro models to evaluate these immunotherapies throughout their development. The use of CAR T-cells on solid tumors has been lagging due to challenges that include tumor heterogeneity, immunosuppressive microenvironments, and the lack of unique tumor antigens that can be recognized by the CAR-T cells. As such, the ability to screen for CAR T-cells (e.g. with CRISPR) that effectively target and kill tumors is an area of active research. Here we describe a method for assessment of T-cell effect and penetration into the multi-cellular 3D tumor spheroids as a proof-of-concept model for in vitro CAR T assays. Spheroids were formed from Hela cell lines in 96-well round bottom plates. We activated the human peripheral blood mononuclear (PBMC) cells with PMA/ionomycin for 6 hours, then treated the spheroids with stimulated PBMC for 72 hours. Samples were imaged with a confocal imager every 2 hours for a 72 hour period. High-content imaging and analysis allowed us to observe and measure phenotypic changes in cancer spheroids and the process of T-cell penetration over a period of 72 hours. To optimize the workflow, we developed an image analysis approach that uses a deep learning-based segmentation model and a machine learning-based classification model to quantify the T-cell induced phenotypic changes in the spheroids using brightfield images. Our results show clear distinct phenotypic changes among different treatment groups and the feasibility of using AI-based analysis workflows to accurately predict the efficacy of T-cells in an in vitro assay. Moreover, the information associated with the penetration of T-cell into the 3D spheroids was explored by calculating the coordinates of the penetrated T-cells and T-cell movements from the nearest edge of the spheroids where the T-cell penetration speed was estimated. Our results show that the stimulated T-cells have much greater penetration speed compared to non-stimulated T-cells. Image analysis also allowed us to measure and quantitate the deterioration of cancer spheroids and changes in their morphology. Overall, our results show that the 3D spheroid models and the high-content analysis workflow may potentially be used as a metric to evaluate the efficacy of cell therapy in vitro. Citation Format: Zhisong Tong, Oksana Sirenko, Misha Bashkurov, Angeline Lim. AI-enabled novel workflow to evaluate T-cell activity in vitro using 3D spheroid models. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 5362.

MRI Evaluation of Soft Tissue Tumors and Tumor-Like Lesions of Extremities.

The current study aimed to evaluate the use of magnetic resonance imaging (MRI) in the diagnosis of extremity soft tissue tumors and tumor-like abnormalities. This prospective observational study of 71 patients with soft tissue lesionsof extremities was conducted at a tertiary hospital and teaching center in western India after obtaining Institutional Ethical Committee (IEC) clearance. All patients underwent an MRI of the region of interest on Siemens Magnetom Vida 3 Tesla MRI (Erlangen, Germany). MRI findings and diagnosis were correlated clinically and with histopathological examination. A total of 71 patients (49 males and 22 females) in the age group of six to 90 years were included in our study. Out of 44 patients with soft tissue tumors, the most common lesion was neurofibroma (18.1%), followed by lipoma and undifferentiatedsarcoma (9.1% each). Liposarcoma, myxoid liposarcoma, giant cell tumor of the tendon, pigmented villonodular synovitis, and schwannoma were seen in 4.5% of patients each. The soft tissue tumor-like lesions were seen in 27 (38%) patients, the most common being slow-flow vascular malformation, which was seen in 9/27 (33%) patients. The second most common pathology was actinomycosis, seen in four (14.8%) patients. Out of 44 patients with soft tissue tumors, 27 (61.4%) were benign and 17 (38.6%) were malignant.Tumor size of more than 5 cm was more commonly seen in malignant tumors (70.5%) than benign tumors (40.7%). The smooth margin was more common in benign tumors (70.3), while most malignant tumors (70.5%) had irregular or lobulated margins.Heterogenous enhancement was more common in malignant tumors (82.3%) than benign tumors (62.9). The odds of a benign histopathological diagnosis for a tumor suspected to be benign by MRI were 93.75 times higher than the odds of a benign histopathological diagnosis for a tumor suspected to be malignant by MRI. MRI is extremely useful in the evaluation of different soft tissue masses and helps in evaluating the characteristics of the masses, their extent and relationship to surrounding structures, and bone destruction, multiplicity, composition, and enhancement pattern. The systematic imaging analysis approach helps to differentiate a benign lesion from a malignant lesion and also in differentiating various soft tissue tumor mimics.

An object-based image analysis approach for comparing tree detection from satellite imagery at different scales; A case study in Sukumba Mali

This paper combines an object-based image analysis (OBIA) approach, with markov-random field based-super resolution mapping (MRF-SRM) technique to extract individual tree objects from satellite imagery. Extraction of individual trees in urban and developing cities using satellite imageries has been quite challenging and a subject of active research with many approaches to it. The study area is a part of Sukumba village located in Koutiala district, Mali. The aim is to evaluate the performance comparison of the combined MRF-SRM approach, which was demonstrated for each combination of scale factor and class separability. Due to varying contrast sensitivity, there exist a general limitation in the spatial distribution of land cover data sets derived from most coarse and fine resolution satellite imageries. A 10 m spatial resolution Sentinel-2 and 2 m spatial resolution Worldview-3 satellite images were used for the study. First, the pixel based MRF-SRM classification technique of Tolpekin and Stein, 2009 was applied on both images. Subsequently, the classified SRM thematic results were partitioned into objects (segments) using region growing segmentation algorithm as post a classification procedure. Finally, validation and comparison of the resulting tree objects was done using an object based image analysis approach to determine existential (TP), extensional (FP) and positional (PA) accuracy. For Sentinel-2 image and out of 1787 reference objects, 1214 (TP) was detected and 573 (FP) was undetected. The PA is 6.504 m out of 10 m, total detection error is 0.467 m and the final detection accuracy is 68%. Similarly, for Worldview-3 image and out of 1787 reference objects, 1772 (TP) was detected and 15 (FP) was undetected. The PA is 1.714 m out of 2 m, total detection error is 0.318 m and the final detection accuracy is 99%. The successful approach used in this study will support the capacity of monitoring agricultural change in data sparse regions and developing countries.

Estimating uncertainty in read-out patterns: Application to controls-based denoising and voxel-based morphometry patterns in neurodegenerative and neuropsychiatric diseases.

Quantifying pathology-related patterns in patient data with pattern expression score (PES) is a standard approach in medical image analysis. In order to estimate the PES error, we here propose to express the uncertainty contained in read-out patterns in terms of the expected squared Euclidean distance between the read-out pattern and the unknown "true" pattern (squared standard error of the read-out pattern, SE2 ). Using SE2 , we predicted and optimized the net benefit (NBe) of the recently suggested method controls-based denoising (CODE) by weighting patterns of nonpathological variance (NPV). Multi-center MRI (1192 patients with various neurodegenerative and neuropsychiatric diseases, 1832 healthy controls) were analysed with voxel-based morphometry. For each pathology, accounting for SE2 , NBe correctly predicted classification improvement and allowed to optimize NPV pattern weights. Using these weights, CODE improved classification performances in all but one analyses, for example, for prediction of conversion to Alzheimer's disease (AUC 0.81 vs. 0.75, p=.01), diagnosis of autism (AUC 0.66 vs. 0.60, p < .001), and of major depressive disorder (AUC 0.62 vs. 0.50, p=.03). We conclude that the degree of uncertainty in a read-out pattern should generally be reported in PES-based analyses and suggest using weighted CODE as a complement to PES-based analyses.

Mapping environmentally sustainable urban development within six US cities through object-based image change analysis of aerial orthoimagery

The rapid expansion of cities and continuous urban population growth underscores a need for sustainable urban development. Sustainable development is that which addresses human needs, contributes to well-being, is economically viable, and utilizes natural resources at a degree sustainable by the surrounding environmental systems. Urban green spaces, green roofs, and solar panels are examples of environmentally sustainable urban development (ESUD), or development that focuses on environmental impact, but also presents the potential to achieve social and economic sustainability. The aim of this study was to map and compare amounts of ESUD c. 2010 and c. 2019 through an object-based image analysis (OBIA) approach using National Agricultural Imagery Program (NAIP) aerial orthoimagery for six mid- to large-size cities in the USA.The results of this study indicate a hybrid OBIA and manual interpretation approach applied to NAIP orthoimagery may allow for reliable mapping and areal estimation of urban green space and green roof changes in urban areas. The reliability of OBIA-only mapping and estimation of areal extents of existing green roofs, and new and existing solar panels, is inconclusive due to low mapping accuracy and coarse spatial resolution of aerial orthoimagery relative to some ESUD features. The three urban study areas in humid continental climate zones (Dfa) were estimated to have greater areal extent of new and existing urban green space and existing green roofs, but less areal extent of new green roofs and existing solar panels compared to the three study areas in humid subtropical climate zones (Cfa).

Combined hyperspectral imaging, monolayer stress microscopy, and S8 image analysis approaches for simultaneously interrogating cellular signals and biomechanics.

Second messenger signals, e.g., Ca2+ and cyclic nucleotides, orchestrate a wide range of cellular events. The methods by which second messenger signals determine specific physiological responses are complex. Recent studies point to the importance of temporal and spatial encoding in determining signal specificity. Studies also indicate the importance of mechanical stimuli, substrate stiffness, and mechanical responses - the "mechanosome" - in regulating physiology. Hence, approaches that probe both chemical and mechanical signals are needed. Here, we report preliminary efforts to combine hyperspectral imaging for second messenger signal measurements, monolayer stress microscopy for mechanical force measurements, and S8 analysis software for quantifying localized signals - specifically, Ca2+ dynamics and mechanical forces in human airway smooth muscle cells (HASMCs). HASMCs were prepared as confluent monolayers on 11 kPa gels with embedded fluorescent microparticles that serve as fiducial markers as well as smaller microparticles to measure deformation (strain). Imaging was performed using a custom excitation-scanning hyperspectral microscope. Hyperspectral images were unmixed to identify signals from cellular fluorescent labels (e.g., CAL 590-AM) and fluorescent microparticles. Images were analyzed to quantify localized force dynamics through monolayer stress microscopy. S8 software was used to identify, track, and quantify spatially-localized Ca2+ activity. Results indicate that localized and transient cellular signals and forces can be quantified and mapped within cell populations. Importantly, these results establish a method for simultaneous interrogation of cellular signals and mechanical forces that may play synergistic roles in regulating downstream cellular physiology in confluent monolayers. This work was supported by NIH P01HL066299, R01HL137030, R01HL058506, and NSF MRI1725937. Drs. Leavesley and Rich disclose financial interest in a university start-up company, SpectraCyte LLC, to commercialize spectral imaging technologies.

DeepFixCX: Explainable privacy‐preserving image compression for medical image analysis

AbstractExplanations of a model's biases or predictions are essential to medical image analysis. Yet, explainable machine learning approaches for medical image analysis are challenged by needs to preserve privacy of patient data, and by current trends in deep learning to use unsustainably large models and large datasets. We propose DeepFixCX for explainable and privacy‐preserving medical image compression that is nimble and performant. We contribute a review of the field and a conceptual framework for simultaneous privacy and explainability via tools of compression. DeepFixCX compresses images without learning by removing or obscuring spatial and edge information. DeepFixCX is ante‐hoc explainable and gives privatized post hoc explanations of spatial and edge bias without accessing the original image. DeepFixCX privatizes images to prevent image reconstruction and mitigate patient re‐identification. DeepFixCX is nimble. Compression can occur on a laptop CPU or GPU to compress and privatize 1700 images per second of size 320 × 320. DeepFixCX enables use of low memory MLP classifiers for vision data; permitting small performance loss gives end‐to‐end MLP performance over 70× faster and batch size over 100× larger. DeepFixCX consistently improves predictive classification performance of a Deep Neural Network (DNN) by 0.02 AUC ROC on Glaucoma and Cervix Type detection datasets, and can improve multi‐label chest x‐ray classification performance in seven of 10 tested settings. In all three datasets, compression to less than 5% of original number of pixels gives matching or improved performance. Our main novelty is to define an explainability versus privacy problem and address it with lossy compression.This article is categorized under: Fundamental Concepts of Data and Knowledge &gt; Explainable AI Commercial, Legal, and Ethical Issues &gt; Security and Privacy Fundamental Concepts of Data and Knowledge &gt; Big Data Mining

Performance Analysis of Graph theory-based Contrast Limited Adaptive Histogram Equalization for Image Enhancement

Nowadays, image enhancement has become a major area of research because of the development of applications that are based on vision.Several digital image processing systems employ such image enhancement strategies with the help of graph theory. As the visibility level in low contrast image features is very less,several image enhancement strategies have been introduced with spatial transformations to enhance image qualityfor improved visualization. Nowadays, image processing plays an important role in the analysis of a patient’s health status and has become extremely popular in medical areas for a wide range of clinical assessments. Generally, medical images contain several complex areas and thereby,few pre-processing approaches are applied to reduce the challenges that occur during different phases of the CAD system. Furthermore, because of external noise interferences, poor illuminating settings as well as other imaging device limitations, the clinical diagnosis becomes a challenging process and medical images do not provide important information for precise categorization. Medical images are available in a variety of applications such as computed tomography, Magnetic Resonance Imaging (MRI), mammography, chest X-ray (CXR), and many more. Only the pixel intensity variations between different areas as well as object boundary information are essential for categorization and must be enhanced simultaneously. As a result, the rate of classification in medical images and intensity are increased so that every object during the analysis can be easily identified. The main goal of any image enhancement process is to enhance the quality of the image by reducing noise and on other hand by using three different algorithms such as Luminance Modulation (LM), Gradient Modulation (GM), and Dynamic Histogram Equalization (DHE). These three algorithms are designed with the help of graph theory for effective preservation of edges, losses, and efficient smoothing and to preserve the basic information without any modifications. Image restoration is also referred to as image enhancement and it is concerned with the precise assessment of real images. Generally, the degradation process is not included in many of the image-enhancement approaches that are already existing. Furthermore, with the application of enhancement techniques, the degradation process for medical images results in some significant performance loss. Several techniques have been proposed and the technique which is examined in this research is image enhancement that is based on histogram which mainly concentrates on equalizing the histogram of values. Histogram Equalization (HE) possesses a few basic properties such as altering spatial patterns as well as intensity which in turn results in significant challenges in medical imaging. As a result,Contrast Limited Adaptive Histogram Equalization (CLAHE) is proposed in this work as a feasible approach for medical image analysis to address the problem. The suggested research work demonstrates that the intensity limiting image enhancement with histogram equalization detects the irregularities in dense mammograms with enhanced quality.

Let’s Unleash the Network Judgment: A Self-Supervised Approach for Cloud Image Analysis

Abstract Accurate cloud type identification and coverage analysis are crucial in understanding the Earth’s radiative budget. Traditional computer vision methods rely on low-level visual features of clouds for estimating cloud coverage or sky conditions. Several handcrafted approaches have been proposed; however, scope for improvement still exists. Newer deep neural networks (DNNs) have demonstrated superior performance for cloud segmentation and categorization. These methods, however, need expert engineering intervention in the preprocessing steps—in the traditional methods—or human assistance in assigning cloud or clear sky labels to a pixel for training DNNs. Such human mediation imposes considerable time and labor costs. We present the application of a new self-supervised learning approach to autonomously extract relevant features from sky images captured by ground-based cameras, for the classification and segmentation of clouds. We evaluate a joint embedding architecture that uses self-knowledge distillation plus regularization. We use two datasets to demonstrate the network’s ability to classify and segment sky images—one with ∼ 85,000 images collected from our ground-based camera and another with 400 labeled images from the WSISEG database. We find that this approach can discriminate full-sky images based on cloud coverage, diurnal variation, and cloud base height. Furthermore, it semantically segments the cloud areas without labels. The approach shows competitive performance in all tested tasks, suggesting a new alternative for cloud characterization.

Stool Image Analysis for Digital Health Monitoring By Smart Toilets

Gastrointestinal (GI) conditions are widespread and significantly impact the quality of life and healthcare. Stool appearance is a valuable GI diagnostic indicator, and with the advent of the Internet of Things (IoT), daily monitoring of excreta from a toilet is emerging as a promising digital health tool. This article describes a stool image analysis approach that classifies two physiologically relevant stool characteristics: 1) form and 2) color for an IoT-based smart toilet. We constructed a stool image data set with 3275 images, spanning all seven types of the Bristol stool-form scale (BSFS), a widely used metric for stool consistency and a variety of colors. We used ground-truth data obtained through the annotation of our data set by two gastroenterologists and developed a stool-color card to standardize the labeling of stool colors. We addressed two limitations associated with the application of computer-vision techniques to a smart-toilet system: 1) uneven separability between different stool-form categories and 2) class imbalance in the data set. We present results on hierarchical convolutional neural network (CNN) architectures for training a stool-form classifier and on perceptual color quantization coupled with machine-learning techniques to optimize the color-feature space for the classification of stool color. We utilized an edge–cloud approach to pursue an optimal balance between accuracy and latency and for the classification of stool form, we achieved a balanced accuracy of 84.4% and 84.2% reduction in latency compared to a cloud-model only. For color classification, the logistic-regression (LR) classifier provided 80.8% balanced accuracy and 47.3% reduction in communication latency.

A Multi-Satellite Mapping Framework for Floating Kelp Forests

Kelp forests provide key habitat on the Pacific Coast of Canada; however, the long-term changes in their distribution and abundance remain poorly understood. With advances in satellite technology, floating kelp forests can now be monitored across large-scale areas. We present a methodological framework using an object-based image analysis approach that enables the combination of imagery from multiple satellites at different spatial resolutions and temporal coverage, to map kelp forests with floating canopy through time. The framework comprises four steps: (1) compilation and quality assessment; (2) preprocessing; (3) an object-oriented classification; and (4) an accuracy assessment. Additionally, the impact of spatial resolution on the detectability of floating kelp forests is described. Overall, this workflow was successful in producing accurate maps of floating kelp forests, with global accuracy scores of between 88% and 94%. When comparing the impact of resolution on detectability, lower resolutions were less reliable at detecting small kelp forests in high slope areas. Based on the analysis, we suggest removing high slope areas (11.4%) from time series analyses using high- to medium-resolution satellite imagery and that error, in this case up to 7%, be considered when comparing imagery at different resolutions in low–mid slope areas through time.

Approaches to standardising the magnetic resonance image analysis of equine tendon lesions.

Low-field magnetic resonance imaging (MRI) has gained increasing importance to monitor equine tendon lesions. Comparing results between studies and cases is hampered, because image analysis approaches vary strongly. This study aimed to improve reliability, comparability and time efficiency of quantitative MRI image analysis. Induced tendon lesions were studied over a 24-week period with 10 follow-up MRI examinations. Signal intensities (SIs) of tendons, tendon lesions, cortical bone and background, as well as lesion cross-sectional areas (CSAs) were measured. Lesion SI standardisation with different formulas was evaluated, using histological findings as reference. Different types of region of interest (ROI) for lesion SI measurement were compared. Lesion CSA measurement at different levels was evaluated, using the calculated total lesion volume as reference. Subjective lesion identification and manual CSA and SI measurements were compared to an automated, algorithm-based approach. Lesion SI standardised using a quotient of lesion and background or cortical bone SI, correlated best with histologically determined lesion severity. Lesion SI in circular ROIs correlated strongly with lesion SI in free-hand whole-lesion ROIs. The level of the maximum lesion CSA shifted over time; the CSA maximum correlated strongly with lesion volume. In sequences with short acquisition time, algorithm-based automated lesion detection showed almost perfect agreement with subjective lesion identification. Automated measurement of CSA and SI was also feasible, with stronger correlation and better agreement with the manually obtained data for the SI than for the CSA. Our study may provide guidance for MRI image analysis of tendon healing. Reliable image analysis can be performed time-efficiently, particularly regarding lesion SI quantification.

A single multi-scale and multi-sourced semi-automated lineament detection technique for detailed structural mapping with applications to geothermal energy exploration

Semi-automated algorithms incorporating multi-sourced datasets into a single analysis are increasingly common, but until now operate at a fixed pixel resolution resulting in multi-sourced methods being limited by the largest input pixel size. Multi-scale lineament detection circumvents this issue and allows increased levels of detail to be captured. We present a semi-automated method using a bottom-up Object-Based Image Analysis approach to map regional lineaments to a high level of detail. The method is applied to onshore light detection and ranging (LiDAR) data and offshore bathymetry around the Land's End Granite (Cornwall, UK). The method uses three different pixel resolutions to extract detailed lineaments across a 700 km 2 area. The granite displays large-scale NW–SE fault zones that are considered analogous to those being targeted as onshore deep geothermal reservoirs (2–5 km in depth). Investigation of the lineaments derived from this study shows along-strike variations from NW–SE orientations within granite to NNW–SSE within slate and reflects structural inheritance of early Variscan structures within Devonian slates. This is furthered by analysing these major structures for reservoir potential. Lineaments proximal to these broadly NW–SE features indicate that a damage zone c. 100–200 m wide is present. These observations provide a preliminary understanding of reservoir characteristics for fault-hosted geothermal systems. Thematic collection: This article is part of the Remote sensing for site investigations on Earth and other planets collection available at: https://www.lyellcollection.org/topic/collections/remote-sensing-for-site-investigations-on-earth-and-other-planets Supplementary material: A description of the OBAI method and additional figures are available at https://doi.org/10.6084/m9.figshare.c.6309629

NNPipe: a neural network pipeline for automated analysis of morphologically diverse catalyst systems

We describe nNPipe for the automated analysis of morphologically diverse catalyst materials. Automated imaging routines and direct-electron detectors have enabled the collection of large data stacks over a wide range of sample positions at high temporal resolution. Simultaneously, traditional image analysis approaches are slow and hence unsuitable for large data stacks and consequently, researchers have progressively turned towards machine learning and deep learning approaches. Previous studies often detail work on morphologically uniform material systems with clearly discernible features, limited workable image sizes and training data that may be biased due to manual labelling. The nNPipe data-processing method consists of two standalone convolutional neural networks that were exclusively trained on multislice image simulations and enables fast analysis of 2048 × 2048 pixel images. Inference performance compared between idealised and real industrial catalytic samples and insights derived from subsequent data analysis are placed into the context of an automated imaging scenario.

Remodelling of the fibre-aggregate structure of collagen gels by cancer-associated fibroblasts: A time-resolved grey-tone image analysis based on stochastic modelling.

Solid tumors consist of tumor cells associated with stromal and immune cells, secreted factors and extracellular matrix (ECM), which together constitute the tumor microenvironment. Among stromal cells, activated fibroblasts, known as cancer-associated fibroblasts (CAFs) are of particular interest. CAFs secrete a plethora of ECM components including collagen and modulate the architecture of the ECM, thereby influencing cancer cell migration. The characterization of the collagen fibre network and its space and time-dependent microstructural modifications is key to investigating the interactions between cells and the ECM. Developing image analysis tools for that purpose is still a challenge because the structural complexity of the collagen network calls for specific statistical descriptors. Moreover, the low signal-to-noise ratio of imaging techniques available for time-resolved studies rules out standard methods based on image segmentation. In this work, we develop a novel approach based on the stochastic modelling of the gel structure and on grey-tone image analysis. The method is then used to study the remodelling of a collagen matrix by migrating breast cancer-derived CAFs in a three-dimensional spheroid model of cellular invasion imaged by time-lapse confocal microscopy. The structure of the collagen at the scale of a few microns consists in regions with high fibre density separated by depleted regions, which can be thought of as aggregates and pores. The approach developped captures this two-scale structure with a clipped Gaussian field model to describe the aggregates-and-pores large-scale structure, and a homogeneous Boolean model to describe the small-scale fibre network within the aggregates. The model parameters are identified by fitting the grey-tone histograms and correlation functions of the images. The method applies to unprocessed grey-tone images, and it can therefore be used with low magnification, noisy time-lapse reflectance images. When applied to the CAF spheroid time-resolved images, the method reveals different matrix densification mechanisms for the matrix in direct contact or far from the cells. We developed a novel and multidisciplinary image analysis approach to investigate the remodelling of fibrillar collagen in a 3D spheroid model of cellular invasion. The specificity of the method is that it applies to the unprocessed grey-tone images, and it can therefore be used with noisy time-lapse reflectance images of non-fluorescent collagen. When applied to the CAF spheroid time-resolved images, the method reveals different matrix densification mechanisms for the matrix in direct contact or far from the cells.

A single-molecule view on intraflagellar transport in C. elegans chemosensory cilia.

In our cells, motor proteins do not work on their own: cargoes are often transported by multiple motors, of the same type, but often also of opposite directionality. To study motor cooperation, we have been focusing on intraflagellar transport (IFT), which is essential for the assembly and maintenance of cilia. In particular, we study the chemosensory cilia of C. elegans. To visualize IFT with live fluorescence microscopy, we generate mutant worms expressing fluorescent versions of the proteins of interest. Our fluorescence and image-analysis approaches allow us to visualize, track and quantify trains of IFT components moving together, as well as individual motors or IFT proteins. In C. elegans, IFT is driven by groups of tens of three different motor proteins: 2 kinesin-2's (the slow kinesin-II and the faster OSM-3), which drive transport of cargo trains from base to tip of the cilium, and IFT dynein, which drives transport back to the base. Cargo trains move in one go from base to tip and from tip to base. Speed and directionality appear to be regulated by motor proteins docking on and off the trains. Here I present new single-molecule data focusing on the entry of IFT components, including motor proteins, into the cilium. We observe that different IFT components attach to IFT trains in a sequential way, both in time and in location. Pooling the localization information of many individual molecules allows mapping of the 3D ultrastructure of the axoneme with a precision of several tens of nanometers, in wild-type as well as mutant strains lacking e.g. motors. These maps provide new insights in ciliary structure at the base, train-docking dynamics and the interplay of kinesin-II and OSM-3. Our results provide important new insights into how motors cooperate to drive intracellular transport.

Fractal algorithms and RGB image processing in scribal and ink identification on an 1819 secret initiation manuscript to the “Philike Hetaereia”

Historical texts incorporate important characteristics that need to be assessed including genre, text structure and content. Often overlooked are characteristics of handwritten manuscripts commonly divided into legibility, readability and aesthetics. To determine the scientific feasibility of classification of handwritten texts an objective approach is developed to describe twenty handwritten pages of an 1819 Greek manuscript, that refers to the initiation to the Greek secret “friendly society” (Philike Hetaereia) organization, established as part of the Greek independence against the Ottoman Turks. It is investigated through a fractal and RGB image analysis approach. Fractal Minkowski Dimension was applied on the handwritten text and the RGB color analysis on the ink and paper and both were used as a non-invasive manner and revealed interesting results. The novel RGB image analysis and the fractal analysis of the manuscript identified respectively, five iron gall inks and four scribes from the ink content and handwritten styles, of the compact five lines text and whole text pages. The novel approach was verified with another old manuscript of known ink pigments, as well as with thirteen known handwritten texts of that period and four prints representing modern and similar period texts substantiating the findings of the novel methods.

Testing the theoretical principles of citizen science in monitoring stream water levels through photo-trap frames

IntroductionMonitoring water levels of ephemeral streams is a difficult yet important task in hydrology, especially when studying minor river flows in remote areas. The installation of flow gauging stations on upstream tributaries is impacted by the lack of economic resources, by accessibility problems and unstable morphological conditions of riverbeds avoiding the implementation of distributed observation networks at large scales. This major challenge in hydrology may be addressed by eventually adopting image-analysis approaches that constitute an effective parsimonious river flow monitoring method, but the demonstration of such techniques is still an open research topic.MethodologyThis study focuses on the testing of a novel technique that employs a white pole “staff gauge” to be photographed using a phototrap (i.e., named stage-cam which is a high-speed camera trigger system). This technology shows to be particularly efficient for observing flood events that represent the most difficult scenario for streamflow monitoring. Furthermore, the testing of this innovative hydrological data-gathering method is performed by adopting citizen science and participatory image analysis to assess the value and effectiveness of non-expert volunteers to operationalize this novel method. Citizen engagement may be essential for supporting distributed flow monitoring supporting large scale image analysis algorithm calibration associated to a continuous series of phototrap images. The Montecalvello watershed, located near Rome, is selected for this pilot case study.ResultsResults of the conducted tests, involving the University of Tuscia student community, are presented toward the demonstration of the effectiveness of citizen science to collect valid quantitative hydrological observations, which may correlate consistently with expert estimates. To better interpret results, the authors consider mean absolute error (MAE) and mean absolute relative error (MARE) as synthetic indices to determine the uncertainties associated to voluntary observations. Low margins of error return positive feedback on the adopted methodology.DiscussionThis research promotes the use of participatory approaches for addressing an actual hydrological monitoring challenge. In addition, it fosters increased citizen knowledge and awareness of the importance and value of hydrological monitoring of small ungauged river basins.