Microscopy Image Data Research Articles

Advanced Cell Classifier: User-Friendly Machine-Learning-Based Software for Discovering Phenotypes in High-Content Imaging Data

High-content, imaging-based screens now routinely generate data on a scale that precludes manual verification and interrogation. Software applying machine learning has become an essential tool to automate analysis, but these methods require annotated examples to learn from. Efficiently exploring large datasets to find relevant examples remains a challenging bottleneck. Here, we present Advanced Cell Classifier (ACC), a graphical software package for phenotypic analysis that addresses these difficulties. ACC applies machine-learning and image-analysis methods to high-content data generated by large-scale, cell-based experiments. It features methods to mine microscopic image data, discover new phenotypes, and improve recognition performance. We demonstrate that these features substantially expedite the training process, successfully uncover rare phenotypes, and improve the accuracy of the analysis. ACC is extensively documented, designed to be user-friendly for researchers without machine-learning expertise, and distributed as a free open-source tool at www.cellclassifier.org.

Temperature‐dependent morphology and characteristic parameters of annealed gold nanolayers

Gold nanostructures were formed when Au nanolayers were deposited on native‐oxide‐covered Si substrates and annealed. To analyze the annealing temperature dependent surface morphology, we calculated the height–height correlation function and the power spectrum density function from the atomic force microscopy image data. When the annealing temperature for 20 nm thick Au layer increased from 500 to 700 °C, the interface width w and the lateral correlation length ξ of the surface increased obviously from 4.8 to 44.0 nm and from 70.5 to 180.9 nm, respectively. On the other hand, the roughness exponent α was a relatively universal parameter (∼0.87). While the morphology of the surfaces was mainly self‐affine, suggesting that the random surface diffusion plays a dominant role in the kinetic roughening process of such an annealed system, the results of the morphological analysis also imply that the self‐affine surface begins to become mounded for higher annealing temperatures.

Low Dimensional Representation of Fisher Vectors for Microscopy Image Classification.

Microscopy image classification is important in various biomedical applications, such as cancer subtype identification, and protein localization for high content screening. To achieve automated and effective microscopy image classification, the representative and discriminative capability of image feature descriptors is essential. To this end, in this paper, we propose a new feature representation algorithm to facilitate automated microscopy image classification. In particular, we incorporate Fisher vector (FV) encoding with multiple types of local features that are handcrafted or learned, and we design a separation-guided dimension reduction method to reduce the descriptor dimension while increasing its discriminative capability. Our method is evaluated on four publicly available microscopy image data sets of different imaging types and applications, including the UCSB breast cancer data set, MICCAI 2015 CBTC challenge data set, and IICBU malignant lymphoma, and RNAi data sets. Our experimental results demonstrate the advantage of the proposed low-dimensional FV representation, showing consistent performance improvement over the existing state of the art and the commonly used dimension reduction techniques.

A curated collection of tissue microarray images and clinical outcome data of prostate cancer patients

Microscopy image data of human cancers provide detailed phenotypes of spatially and morphologically intact tissues at single-cell resolution, thus complementing large-scale molecular analyses, e.g., next generation sequencing or proteomic profiling. Here we describe a high-resolution tissue microarray (TMA) image dataset from a cohort of 71 prostate tissue samples, which was hybridized with bright-field dual colour chromogenic and silver in situ hybridization probes for the tumour suppressor gene PTEN. These tissue samples were digitized and supplemented with expert annotations, clinical information, statistical models of PTEN genetic status, and computer source codes. For validation, we constructed an additional TMA dataset for 424 prostate tissues, hybridized with FISH probes for PTEN, and performed survival analysis on a subset of 339 radical prostatectomy specimens with overall, disease-specific and recurrence-free survival (maximum 167 months). For application, we further produced 6,036 image patches derived from two whole slides. Our curated collection of prostate cancer data sets provides reuse potential for both biomedical and computational studies.

Detecting ordered small molecule drug aggregates in live macrophages: a multi-parameter microscope image data acquisition and analysis strategy.

Following prolonged administration, certain orally bioavailable but poorly soluble small molecule drugs are prone to precipitate out and form crystal-like drug inclusions (CLDIs) within the cells of living organisms. In this research, we present a quantitative multi-parameter imaging platform for measuring the fluorescence and polarization diattenuation signals of cells harboring intracellular CLDIs. To validate the imaging system, the FDA-approved drug clofazimine (CFZ) was used as a model compound. Our results demonstrated that a quantitative multi-parameter microscopy image analysis platform can be used to study drug sequestering macrophages, and to detect the formation of ordered molecular aggregates formed by poorly soluble small molecule drugs in animals.

Automated neuron tracing using probability hypothesis density filtering.

The functionality of neurons and their role in neuronal networks is tightly connected to the cell morphology. A fundamental problem in many neurobiological studies aiming to unravel this connection is the digital reconstruction of neuronal cell morphology from microscopic image data. Many methods have been developed for this, but they are far from perfect, and better methods are needed. Here we present a new method for tracing neuron centerlines needed for full reconstruction. The method uses a fundamentally different approach than previous methods by considering neuron tracing as a Bayesian multi-object tracking problem. The problem is solved using probability hypothesis density filtering. Results of experiments on 2D and 3D fluorescence microscopy image datasets of real neurons indicate the proposed method performs comparably or even better than the state of the art. Software implementing the proposed neuron tracing method was written in the Java programming language as a plugin for the ImageJ platform. Source code is freely available for non-commercial use at https://bitbucket.org/miroslavradojevic/phd . meijering@imagescience.org. Supplementary data are available at Bioinformatics online.

A Bayesian Topological Framework for the Identification and Reconstruction of Subcellular Motion

Microscopy imaging allows detailed observations of intracellular movements and the acquisition of large datasets that can be fully analyzed only by automated algorithms. Here, we develop a computational method for the automatic identification and reconstruction of trajectories followed by subcellular particles captured in microscopy image data. The method operates on stacks of raw image data and computes the complete set of contained trajectories. The method utilizes topological data analysis and standard image processing techniques and makes no assumptions about the underlying dynamics besides continuity. We test the developed method successfully against artificial and experimental datasets. Application of the method on the experimental data reveals good agreement with manual tracking and benchmarking yields performance scores competitive to the existing state-of-the-art tracking methods.

3D texture analysis for classification of second harmonic generation images of human ovarian cancer.

Remodeling of the collagen architecture in the extracellular matrix (ECM) has been implicated in ovarian cancer. To quantify these alterations we implemented a form of 3D texture analysis to delineate the fibrillar morphology observed in 3D Second Harmonic Generation (SHG) microscopy image data of normal (1) and high risk (2) ovarian stroma, benign ovarian tumors (3), low grade (4) and high grade (5) serous tumors, and endometrioid tumors (6). We developed a tailored set of 3D filters which extract textural features in the 3D image sets to build (or learn) statistical models of each tissue class. By applying k-nearest neighbor classification using these learned models, we achieved 83–91% accuracies for the six classes. The 3D method outperformed the analogous 2D classification on the same tissues, where we suggest this is due the increased information content. This classification based on ECM structural changes will complement conventional classification based on genetic profiles and can serve as an additional biomarker. Moreover, the texture analysis algorithm is quite general, as it does not rely on single morphological metrics such as fiber alignment, length, and width but their combined convolution with a customizable basis set.

Analysis of confocal microscopy image data of Physcomitrella chloroplasts to reveal adaptation principles leading to structural stability at the nanoscale

AbstractProteins of the FtsZ (filamentous temperature‐sensitive Z) family establish complex polymeric spatial patterns in plastids of the moss Physcomitrella patens. These structures represent a "plastoskeleton" that might contribute to plastid shape and stability. The aim of this work is to develop computer models of FtsZ network connectivity and dynamics. These computer models are developed from green fluorescent protein (GFP) images obtained by Prof. Reski's Plant Biotechnology group at the University of Freiburg by means of laser scanning confocal microscopy. Subsequently, structural reconstruction techniques based on cellular automata algorithm are applied on the raw image data to compensate for any information that is lost in the process of imaging. Segmentation of the processed data will result in the computer model of the targeted structures, which are used for further analysis. Model generation and experimental testing in the moss system is conducted in an iterative process with the aim of both forward and reverse biomimetics. A developed image processing algorithm with the aim of quantification of structural characteristics is utilized to extract structural features of the FtsZ network in Physcomitrella enabling us a statistical analysis on the extracted data. (© 2016 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Automated Neuron Tracing Methods: An Updated Account.

The reconstruction of neuron morphology allows to investigate how the brain works, which is one of the foremost challenges in neuroscience. This process aims at extracting the neuronal structures from microscopic imaging data. The great advances in microscopic technologies have made a huge amount of data available at the micro-, or even lower, resolution where manual inspection is time consuming, prone to error and utterly impractical. This has motivated the development of methods to automatically trace the neuronal structures, a task also known as neuron tracing. This paper surveys the latest neuron tracing methods available in the scientific literature as well as a selection of significant older papers to better place these proposals into context. They are categorized into global processing, local processing and meta-algorithm approaches. Furthermore, we point out the algorithmic components used to design each method and we report information on the datasets and the performance metrics used.

Abstract 1671: Local release of combretastatin A-4 from NIR-light activatable prodrugs overcomes areal and temporal limitations of photodynamic therapy

Abstract A unique prodrug strategy for treating localized cancers, in which NIR light-illuminated prodrug effectively ablates tumors through the combined effects of photodynamic therapy (i.e., singlet oxygen [SO]) and locally released anticancer drugs has been proposed. Due to short distance of action (< 0.04 μs) and short lifetime (< 0.02 μm) of SO, direct damage of PDT is both areally and temporally limited. We hypothesized that the locally released anticancer drugs would overcome the areal and temporal limits of SO. Near IR-activatable prodrug of combretastatin A-4 (CA4), Pc-(L-CA4)2, and its pseudo-prodrug, Pc-(NCL-CA4)2, were evaluated in vitro and in vivo. After partial illumination of a 24 well, all the cells in the prodrug-treated well were killed by the released CA4. Limited areal damage was observed in the pseudo-prodrug-treated wells. A time-dependent cell survival study revealed more extensive cell death in the prodrug-treated cells, due to the sustained damage from the released CA4. Cell cycle analysis and microscopic imaging data demonstrated the typical damage patterns of CA4 in the prodrug-treated cells. A time-dependent histological study showed that prodrug-treated tumors lacked mitotic bodies. The prodrug caused broader and more long-lasting tumor size reduction than did the pseudo-prodrug. These data consistently support that the released CA4 overcomes the areal and temporal limits of SO, providing far superior antitumor effects. Citation Format: Pallavi Rajaputra, Moses Bio, Gregory Nkepang, Pritam Thapa, Sukyung Woo, Youngjae You. Local release of combretastatin A-4 from NIR-light activatable prodrugs overcomes areal and temporal limitations of photodynamic therapy. [abstract]. In: Proceedings of the 107th Annual Meeting of the American Association for Cancer Research; 2016 Apr 16-20; New Orleans, LA. Philadelphia (PA): AACR; Cancer Res 2016;76(14 Suppl):Abstract nr 1671.

The influences of different spatial resolutions on the characteristics of T2 relaxation times in articular cartilage: A coarse-graining study of the microscopic magnetic resonance imaging data.

Microscopic magnetic resonance imaging (µMRI) T2 data from canine cartilage at different tibial locations were analyzed to investigate the influences of spatial resolution and pixel position on the T2 sensitivity to osteoarthritis (OA). Five experimental factors were investigated: inaccurate pixel position, different pixel resolutions, different specimen orientations in the magnetic field, topographical variations over the tibial surface, and different OA stages. A number of significant trends were identified in this analysis, which shows the subtle but substantial influences to our abilities of detecting OA due to T2 changes. In particular, any deviation in locating the cartilage pixels may result in erratic values near the cartilage surface. Significant differences were found in T2 values between nearly any two comparison-groups under all resolutions both in the meniscus-covered and -uncovered areas, which were also showed interaction between the OA degradation stages. This multiresolution project should help to improve the detection sensitivities of MRI toward cartilage degeneration. Microsc. Res. Tech. 79:754-765, 2016. © 2016 Wiley Periodicals, Inc.

(Invited) Upconverting Fluoride Nanocrystals - Synthesis and Optical Properties

Over the past decade, high-quality lanthanide doped upconverting nanocrystals (UPCNCs) have been successfully synthesized with the rapid development of nanotechnology. Several materials (i.e. CaF2, NaYF4, NaGdF4) have been successfully applied so far as potential matrix for Yb3+-RE3+combinations, where the low crystal field symmetry and low phonon frequency are the main factors defining the best matrix selection. These materials have already found several potential applications in biology, medicine as well as in photovoltaics and photocatalysis – as the most common examples. However, to change the commercial potential of UPCNCs to real devices still a several problems must be solved, where week emission quantum efficiency (EQE typically < 2%) is one of the most important one. The EQE is directly related to other challenges in this field: growth of small size <10 nm UPCNCs, UPCNCs shape and architecture control and control of UPCNCs surface properties. To solve all abovementioned problems the correlation between technological, morphological and physical parameters is needed. This is however a difficult task because the system itself is very complicated and consist number of correlated parameters. Thus, still a better understanding of physical processes responsible for the optical properties of UPCNCs in needed to bring this fascinating material to commercial use. In this work, we will discuss synthesis and optical properties of NaGdF4-based up-converting nanocrystals. We will discuss how to control size, shape and core-shell architecture of lanthanide doped NaGdF4 nanocrystals focusing mainly on Yb-Er co-doping. We will discuss how doping influencing the nanocrystals size, shape and their optical properties (Fig.1.). This will be done based on experimental results obtained from photoluminescence (PL) measurements, PL-excitation spectroscopy, PL-decay measurements, microscopic imaging and structural data obtained from HRTEM and XRD. Our conclusions will be supported by results obtained from numerical Monte Carlo modeling used to model the kinetic properties of nanocrystals emission and to model the ions distribution within the nanocrystals volume. Finally, based on our work on nanocrystals surface functionalization and their toxicity tests we will briefly discuss the potential use of UPCNCs in biology and medicine. Figure 1. ß-NaGdF4:Yb, Er nanocrystals with different shapes and architectures. Figure 1

Investigating the usage of point spread functions in point source and microsphere localization.

Using a point spread function (PSF) to localize a point-like object, such as a fluorescent molecule or microsphere, represents a common task in single molecule microscopy image data analysis. The localization may differ in purpose depending on the application or experiment, but a unifying theme is the importance of being able to closely recover the true location of the point-like object with high accuracy. We present two simulation studies, both relating to the performance of object localization via the maximum likelihood fitting of a PSF to the object's image. In the first study, we investigate the integration of the PSF over an image pixel, which represents a critical part of the localization algorithm. Specifically, we explore how the fineness of the integration affects how well a point source can be localized, and find the use of too coarse a step size to produce location estimates that are far from the true location, especially when the images are acquired at relatively low magnifications. We also propose a method for selecting an appropriate step size. In the second study, we investigate the suitability of the common practice of using a PSF to localize a microsphere, despite the mismatch between the microsphere's image and the fitted PSF. Using criteria based on the standard errors of the mean and variance, we find the method suitable for microspheres up to 1 μm and 100 nm in diameter, when the localization is performed, respectively, with and without the simultaneous estimation of the width of the PSF.

Iminosugar-Cyclopeptoid Conjugates Raise Multivalent Effect in Glycosidase Inhibition at Unprecedented High Levels.

A series of cyclopeptoid-based iminosugar clusters has been evaluated to finely probe the ligand content-dependent increase in α-mannosidase inhibition. This study led to the largest binding enhancement ever reported for an enzyme inhibitor (up to 4700-fold on a valency-corrected basis), which represents a substantial advance over the multivalent glycosidase inhibitors previously reported. Electron microscopy imaging and analytical data support, for the best multivalent effects, the formation of a strong chelate complex in which two mannosidase molecules are cross-linked by one inhibitor.

Anticancer drug released from near IR-activated prodrug overcomes spatiotemporal limits of singlet oxygen

Photodynamic therapy (PDT) is a cancer treatment modality where photosensitizer (PS) is activated by visible and near IR light to produce singlet oxygen (1O2). However, 1O2 has a short lifetime (<40ns) and cannot diffuse (<20nm) beyond the cell diameter (e.g., ∼1800nm). Thus, 1O2 damage is both spatially and temporally limited and does not produce bystander effect. In a heterogeneous tumor, cells escaping 1O2 damage can regrow after PDT treatment. To overcome these limitations, we developed a prodrug concept (PS–L–D) composed of a photosensitizer (PS), an anti-cancer drug (D), and an 1O2-cleavable linker (L). Upon illumination of the prodrug, 1O2 is generated, which damages the tumor and also releases anticancer drug. The locally released drug could cause spatially broader and temporally sustained damage, killing the surviving cancer cells after the PDT damage. In our previous report, we presented the superior activity of our prodrug of CA4 (combretastatin A-4), Pc-(L-CA4)2, compared to its non-cleavable analog, Pc-(NCL-CA4)2, that produced only PDT effects. Here, we provide clear evidence demonstrating that the released anticancer drug, CA4, indeed damages the surviving cancer cells over and beyond the spatial and temporal limits of 1O2. In the limited light illumination experiment, cells in the entire well were killed due to the effect of released anti-cancer drug, whereas only a partial damage was observed in the pseudo-prodrug treated wells. A time-dependent cell survival study showed more cell death in the prodrug-treated cells due to the sustained damage by the released CA4. Cell cycle analysis and microscopic imaging data demonstrated the typical damage patterns by CA4 in the prodrug treated cells. A time-dependent histological study showed that prodrug-treated tumors lacked mitotic bodies, and the prodrug caused broader and sustained tumor size reduction compared to those seen in the tumors treated with the pseudo-prodrug. This data consistently support that the released CA4 overcomes the spatiotemporal limitations of 1O2, providing far superior antitumor effect.

Reaction paths of phosphine dissociation on silicon (001).

Using density functional theory and guided by extensive scanning tunneling microscopy (STM) image data, we formulate a detailed mechanism for the dissociation of phosphine (PH3) molecules on the Si(001) surface at room temperature. We distinguish between a main sequence of dissociation that involves PH2+H, PH+2H, and P+3H as observable intermediates, and a secondary sequence that gives rise to PH+H, P+2H, and isolated phosphorus adatoms. The latter sequence arises because PH2 fragments are surprisingly mobile on Si(001) and can diffuse away from the third hydrogen atom that makes up the PH3 stoichiometry. Our calculated activation energies describe the competition between diffusion and dissociation pathways and hence provide a comprehensive model for the numerous adsorbate species observed in STM experiments.

Acceleration effect of copper(II) ions on the rate of citrate synthesis of gold nanoparticles

The gold nanoparticles (GNPs) were obtained by modification of the classical Turkevich citrate method by the addition of copper(II) ions in the initial reaction mixture. We studied the kinetics of the nanoparticles growth with or without copper(II) ions, synthesized at two different temperatures. The kinetic curves obtained by means of UV–vis spectroscopy from the plasmon maxima demonstrated the acceleration effects of the copper(II) ions on nucleation and growth of the GNPs. These effects were also confirmed from the Atomic Force Microscope (AFM) imaging data where the AFM was applied as a tool for studying the kinetics of the GNPs’ growth. From the analysis of AFM images in the course of the synthesis process were obtained the sizes (i.e., diameters) of the GNPs and the GNPs’ growth profiles were fitted with the Finke–Watzky (F–W) kinetic model. From the experimental data and the theoretical fits we found that the citrate reduction of the tetrachloroauric acid was accelerated by the addition of copper(II) ions as well as the GNPs’ growth was accelerated following the F–W kinetics model. Using Transmission Electron Microscopy (TEM) we compared the morphology and the size distribution of the nanoparticles synthesized by the citrate method with or without copper(II) ions. The results showed that the addition of copper(II) ions in the initial reaction mixture of the Turkevich citrate synthesis let to acceleration of both the nucleation and the growth processes following the F–W kinetic model.

Rapid and Semi-automated Extraction of Neuronal Cell Bodies and Nuclei from Electron Microscopy Image Stacks.

Connectomics-the study of how neurons wire together in the brain-is at the forefront of modern neuroscience research. However, many connectomics studies are limited by the time and precision needed to correctly segment large volumes of electron microscopy (EM) image data. We present here a semi-automated segmentation pipeline using freely available software that can significantly decrease segmentation time for extracting both nuclei and cell bodies from EM image volumes.

Recent advances in holographic 3D particle tracking

Particle tracking is a fundamental technique for investigating a variety of biophysical processes, from intracellular dynamics to the characterization of cell motility and migration. However, observing three-dimensional (3D) trajectories of particles is in general a challenging task in classical microscopy owing to the limited imaging depth of field of commercial optical microscopes, which represents a serious drawback for the analysis of time-lapse microscopy image data. Therefore, numerous automated particle-tracking approaches have been developed by many research groups around the world. Recently, digital holography (DH) in microscopy has rapidly gained credit as one of the elective techniques for these applications, mainly due to the uniqueness of the DH to provide a posteriori quantitative multiple refocusing capability and phase-contrast imaging. Starting from this paradigm, a huge amount of 3D holographic tracking approaches have been conceived and investigated for applications in various branches of science, including optofluids, microfluidics, biomedical microscopy, cell mechano-trasduction, and cell migration. Since a wider community of readers could be interested in such a review, i.e., not only scientists working in the fields of optics and photonics but also users of particle-tracking tools, it should be very beneficial to provide a complete review of state-of-the-art holographic 3D particle-tracking methods and their applications in bio-microfluidics.