Automated Image Analysis Framework Research Articles

Morning Glory Flower Detection in Aerial Images Using Semi-Supervised Segmentation with Gaussian Mixture Models

The invasive morning glory, Ipomoea purpurea (Convolvulaceae), poses a mounting challenge in vineyards by hindering grape harvest and as a secondary host of disease pathogens, necessitating advanced detection and control strategies. This study introduces a novel automated image analysis framework using aerial images obtained from a small fixed-wing unmanned aircraft system (UAS) and an RGB camera for the large-scale detection of I. purpurea flowers. This study aimed to assess the sampling fidelity of aerial detection in comparison with the actual infestation measured by ground validation surveys. The UAS was systematically operated over 16 vineyard plots infested with I. purpurea and another 16 plots without I. purpurea infestation. We used a semi-supervised segmentation model incorporating a Gaussian Mixture Model (GMM) with the Expectation-Maximization algorithm to detect and count I. purpurea flowers. The flower detectability of the GMM was compared with that of conventional K-means methods. The results of this study showed that the GMM detected the presence of I. purpurea flowers in all 16 infested plots with 0% for both type I and type II errors, while the K-means method had 0% and 6.3% for type I and type II errors, respectively. The GMM and K-means methods detected 76% and 65% of the flowers, respectively. These results underscore the effectiveness of the GMM-based segmentation model in accurately detecting and quantifying I. purpurea flowers compared with a conventional approach. This study demonstrated the efficiency of a fixed-wing UAS coupled with automated image analysis for I. purpurea flower detection in vineyards, achieving success without relying on data-driven deep-learning models.

Prediction of Bone Marrow Biopsy Results From MRI in Multiple Myeloma Patients Using Deep Learning and Radiomics.

In multiple myeloma and its precursor stages, plasma cell infiltration (PCI) and cytogenetic aberrations are important for staging, risk stratification, and response assessment. However, invasive bone marrow (BM) biopsies cannot be performed frequently and multifocally to assess the spatially heterogenous tumor tissue. Therefore, the goal of this study was to establish an automated framework to predict local BM biopsy results from magnetic resonance imaging (MRI). This retrospective multicentric study used data from center 1 for algorithm training and internal testing, and data from center 2 to 8 for external testing. An nnU-Net was trained for automated segmentation of pelvic BM from T1-weighted whole-body MRI. Radiomics features were extracted from these segmentations, and random forest models were trained to predict PCI and the presence or absence of cytogenetic aberrations. Pearson correlation coefficient and the area under the receiver operating characteristic were used to evaluate the prediction performance for PCI and cytogenetic aberrations, respectively. A total of 672 MRIs from 512 patients (median age, 61 years; interquartile range, 53-67 years; 307 men) from 8 centers and 370 corresponding BM biopsies were included. The predicted PCI from the best model was significantly correlated (P ≤ 0.01) to the actual PCI from biopsy in all internal and external test sets (internal test set: r = 0.71 [0.51, 0.83]; center 2, high-quality test set: r = 0.45 [0.12, 0.69]; center 2, other test set: r = 0.30 [0.07, 0.49]; multicenter test set: r = 0.57 [0.30, 0.76]). The areas under the receiver operating characteristic of the prediction models for the different cytogenetic aberrations ranged from 0.57 to 0.76 for the internal test set, but no model generalized well to all 3 external test sets. The automated image analysis framework established in this study allows for noninvasive prediction of a surrogate parameter for PCI, which is significantly correlated to the actual PCI from BM biopsy.

Hybrid CNN and XGBoost Model Tuned by Modified Arithmetic Optimization Algorithm for COVID-19 Early Diagnostics from X-ray Images

Developing countries have had numerous obstacles in diagnosing the COVID-19 worldwide pandemic since its emergence. One of the most important ways to control the spread of this disease begins with early detection, which allows that isolation and treatment could perhaps be started. According to recent results, chest X-ray scans provide important information about the onset of the infection, and this information may be evaluated so that diagnosis and treatment can begin sooner. This is where artificial intelligence collides with skilled clinicians’ diagnostic abilities. The suggested study’s goal is to make a contribution to battling the worldwide epidemic by using a simple convolutional neural network (CNN) model to construct an automated image analysis framework for recognizing COVID-19 afflicted chest X-ray data. To improve classification accuracy, fully connected layers of simple CNN were replaced by the efficient extreme gradient boosting (XGBoost) classifier, which is used to categorize extracted features by the convolutional layers. Additionally, a hybrid version of the arithmetic optimization algorithm (AOA), which is also developed to facilitate proposed research, is used to tune XGBoost hyperparameters for COVID-19 chest X-ray images. Reported experimental data showed that this approach outperforms other state-of-the-art methods, including other cutting-edge metaheuristics algorithms, that were tested in the same framework. For validation purposes, a balanced X-ray images dataset with 12,000 observations, belonging to normal, COVID-19 and viral pneumonia classes, was used. The proposed method, where XGBoost was tuned by introduced hybrid AOA, showed superior performance, achieving a classification accuracy of approximately 99.39% and weighted average precision, recall and F1-score of 0.993889, 0.993887 and 0.993887, respectively.

Deep learning-based accurate grapevine inflorescence and flower quantification in unstructured vineyard images acquired using a mobile sensing platform

Early grapevine yield forecasting at satisfactory accuracy is among the major trends in precision viticulture research. Conventionally, yield is estimated manually through extrapolation right before harvest, which is mostly inaccurate, requires considerable labor as well as resources, and is often destructive. The number of flowers per vine is one of the main determinants of grapevine yield and can be used as an early indicator for viticulture yield forecasting. In the present study, a non-invasive, automated image analysis framework was proposed for quantifying flowers in unstructured RGB images of grapevine canopies. Images were automatically acquired under field conditions at night, using a mobile sensing platform equipped with artificial illumination. Due to the small shape and dense distribution of individual flowers (a few hundred per inflorescence) and the similar hue of all plant organs in the fore- and background, efficient flower quantification in images is challenging. To overcome this, we adopted a two-step segmentation approach in our algorithm. First, image regions containing inflorescences were recognized and extracted by segmenting each instance of inflorescences in the image using a mask region-based convolutional neural network (Mask R-CNN), followed by test-time augmentation post-processing to achieve high accuracy. Finally, individual flowers in the extracted inflorescences were detected and quantified using another Mask R-CNN model preceded by a contrast enhancement pre-processing operation. For efficient segmentation and quantification of individual flowers, a high-resolution full image was split down into smaller patches and processed in multiple iterations during inference. The algorithm yielded significant performances, with F1 score values of 0.943 and 0.903 for inflorescence segmentation and single flower detection tasks, respectively, against a test set of 75 images from three different cultivars. A determination coefficient (R2) of 0.98 and a normalized root mean square error of 12.24% were obtained in the test set between the automatic flower number quantification and manual counts. In conclusion, the proposed algorithm constitutes a promising approach for automatically predicting yield potential in the early stages of grapevine development, and it can be used for objective monitoring and optimal management of commercial vineyards.

Besides stem canker severity, oilseed rape host genotype matters for the production of Leptosphaeria maculans fruit bodies

For fungal cyclic epidemics on annual crops, the pathogen carry-over is an important but poorly documented step. Plant resistance affects the pathogen development within the epidemics but we lack data on the inter-annual transmission of inoculum. For Leptosphaeria maculans on 15 oilseed rape genotypes in field during 4 growing seasons, stem canker severity was visually scored at harvest. The number of fruit bodies produced on incubated stubble was quantified using an automated image analysis framework. Our results confirm that fruit body production increases with disease severity and is significantly affected by host genotype and nitrogen supply. Tracking individual stems through incubation, we confirm for the first time that the oilseed rape genotype has a direct effect on inoculum production, not only disease severity. This major effect of genotype on inoculum carry-over should be taken into account in models of varietal deployment strategies.

Volumetric growth tracking of patient-derived cancer organoids using optical coherence tomography

Patient-derived cancer organoids (PCOs) are in vitro organotypic models that reflect in vivo drug response, thus PCOs are an accessible model for cancer drug screening in a clinically relevant timeframe. However, current methods to assess the response of PCOs are limited. Here, a custom swept-source optical coherence tomography (OCT) system was used to rapidly evaluate volumetric growth and drug response in PCOs. This system was optimized for an inverted imaging geometry to enable high-throughput imaging of PCOs. An automated image analysis framework was developed to perform 3D single-organoid tracking of PCOs across multiple time points over 48 hours. Metabolic inhibitors and cancer therapies decreased PCOs volumetric growth rate compared to control PCOs. Single-organoid tracking improved sensitivity to drug treatment compared to a pooled analysis of changes in organoid volume. OCT provided a more accurate assessment of organoid volume compared to a volume estimation method based on 2D projections. Single-organoid tracking with OCT also identified heterogeneity in drug response between solid and hollow PCOs. This work demonstrates that OCT and 3D single-organoid tracking are attractive tools to monitor volumetric growth and drug response in PCOs, providing rapid, non-destructive methods to quantify heterogeneity in PCOs.

Patient-derived cancer organoid tracking with wide-field one-photon redox imaging to assess treatment response.

.Significance: Accessible tools are needed for rapid, non-destructive imaging of patient-derived cancer organoid (PCO) treatment response to accelerate drug discovery and streamline treatment planning for individual patients.Aim: To segment and track individual PCOs with wide-field one-photon redox imaging to extract morphological and metabolic variables of treatment response.Approach: Redox imaging of the endogenous fluorophores, nicotinamide dinucleotide (NADH), nicotinamide dinucleotide phosphate (NADPH), and flavin adenine dinucleotide (FAD), was used to monitor the metabolic state and morphology of PCOs. Redox imaging was performed on a wide-field one-photon epifluorescence microscope to evaluate drug response in two colorectal PCO lines. An automated image analysis framework was developed to track PCOs across multiple time points over 48 h. Variables quantified for each PCO captured metabolic and morphological response to drug treatment, including the optical redox ratio (ORR) and organoid area.Results: The ORR (NAD(P)H/(FAD + NAD(P)H)) was independent of PCO morphology pretreatment. Drugs that induced cell death decreased the ORR and growth rate compared to control. Multivariate analysis of redox and morphology variables identified distinct PCO subpopulations. Single-organoid tracking improved sensitivity to drug treatment compared to pooled organoid analysis.Conclusions: Wide-field one-photon redox imaging can monitor metabolic and morphological changes on a single organoid-level, providing an accessible, non-destructive tool to screen drugs in patient-matched samples.

Automated Myocardial Wall Motion Classification using Handcrafted Features vs a Deep CNN-based mapping.

Compared to other modalities such as computed tomography or magnetic resonance imaging, the appearance of ultrasound images is highly dependent on the expertise of the sonographer or clinician making the image acquisition, as well as the machine used, making it a challenge to analyze due to the frequent presence of artefacts, missing boundaries, attenuation, shadows, and speckle. In addition, manual contouring of the epicardial and endocardial walls exhibits large inconsistencies and variations as it is strongly dependent on the sonographer's training and expertise. Hence, in this paper we propose a fully automated image analysis framework to ultimately perform wall motion abnormality classification in 2D+T images. We explore both traditional Random Forests classification with handcrafted features and spatio-temporal hierarchical aggregation of information with a deep learning CNN-based approach. Regarding the later classifier, we also investigate the effect of local phase information retrieval through the use of Feature Asymmetry (FA), and demonstrate that pre-processing videos with FA enables the spatio-temporal CNN to better discover relevant left ventricle endocardial abstractions from low-level features to high-level representations automatically.

An automated image analysis framework for segmentation and division plane detection of single live Staphylococcus aureus cells which can operate at millisecond sampling time scales using bespoke Slimfield microscopy

Staphylococcus aureus is an important pathogen, giving rise to antimicrobial resistance in cell strains such as Methicillin Resistant S. aureus (MRSA). Here we report an image analysis framework for automated detection and image segmentation of cells in S. aureus cell clusters, and explicit identification of their cell division planes. We use a new combination of several existing analytical tools of image analysis to detect cellular and subcellular morphological features relevant to cell division from millisecond time scale sampled images of live pathogens at a detection precision of single molecules. We demonstrate this approach using a fluorescent reporter GFP fused to the protein EzrA that localises to a mid-cell plane during division and is involved in regulation of cell size and division. This image analysis framework presents a valuable platform from which to study candidate new antimicrobials which target the cell division machinery, but may also have more general application in detecting morphologically complex structures of fluorescently labelled proteins present in clusters of other types of cells.

Novel Methods for Microglia Segmentation, Feature Extraction, and Classification.

Segmentation and analysis of histological images provides a valuable tool to gain insight into the biology and function of microglial cells in health and disease. Common image segmentation methods are not suitable for inhomogeneous histology image analysis and accurate classification of microglial activation states has remained a challenge. In this paper, we introduce an automated image analysis framework capable of efficiently segmenting microglial cells from histology images and analyzing their morphology. The framework makes use of variational methods and the fast-split Bregman algorithm for image denoising and segmentation, and of multifractal analysis for feature extraction to classify microglia by their activation states. Experiments show that the proposed framework is accurate and scalable to large datasets and provides a useful tool for the study of microglial biology.

Automated image analysis framework for high-throughput determination of grapevine berry sizes using conditional random fields

The berry size is one of the most important fruit traits in grapevine breeding. Non-invasive, image-based phenotyping promises a fast and precise method for the monitoring of the grapevine berry size. In the present study an automated image analyzing framework was developed in order to estimate the size of grapevine berries from images in a high-throughput manner. The framework includes (i) the detection of circular structures which are potentially berries and (ii) the classification of these into the class ‘berry’ or ‘non-berry’ by utilizing a conditional random field. The approach used the concept of a one-class classification, since only the target class ‘berry’ is of interest and needs to be modeled. Moreover, the classification was carried out by using an automated active learning approach, i.e. no user interaction is required during the classification process and in addition, the process adapts automatically to changing image conditions, e.g. illumination or berry color. The framework was tested on three datasets consisting in total of 139 images. The images were taken in an experimental vineyard at different stages of grapevine growth according to the BBCH scale. The mean berry size of a plant estimated by the framework correlates with the manually measured berry size by 0.88.

Domain-Specific Image Analysis for Cervical Neoplasia Detection Based on Conditional Random Fields

This paper presents a domain-specific automated image analysis framework for the detection of pre-cancerous and cancerous lesions of the uterine cervix. Our proposed framework departs from previous methods in that we include domain-specific diagnostic features in a probabilistic manner using conditional random fields. Likewise, we provide a novel window-based performance assessment scheme for 2D image analysis which addresses the intrinsic problem of image misalignment. Image regions corresponding to different tissue types are indentified for the extraction of domain-specific anatomical features. The unique optical properties of each tissue type and the diagnostic relationships between neighboring regions are incorporated in the proposed conditional random field model. The validity of our method is examined using clinical data from 48 patients, and its diagnostic potential is demonstrated by a performance comparison with expert colposcopy annotations, using histopathology as the ground truth. The proposed automated diagnostic approach can support or potentially replace conventional colposcopy, allow tissue specimen sampling to be performed in a more objective manner, and lower the number of cervical cancer cases in developing countries by providing a cost effective screening solution in low-resource settings.