Flow Imaging Microscopy Research Articles

Stain-Free Approach to Determine and Monitor Cell Heath Using Supervised and Unsupervised Image-Based Deep Learning

Cell-based medicinal products (CBMPs) are a growing class of therapeutics that promise new treatments for complex and rare diseases. Given the inherent complexity of the whole human cells comprising CBMPs, there is a need for robust and fast analytical methods for characterization, process monitoring, and quality control (QC) testing during their manufacture. Existing techniques to evaluate and monitor cell quality typically constitute labor-intensive, expensive, and highly specific staining assays. In this work, we combine image-based deep learning with flow imaging microscopy (FIM) to predict cell health metrics using cellular morphology “fingerprints” extracted from images of unstained Jurkat cells (immortalized human T-lymphocyte cells). A supervised (i.e., algorithm trained with human-generated labels for images) fingerprinting algorithm, trained on images of unstained healthy and dead cells, provides a robust stain-free, non-invasive, and non-destructive method for determining cell viability. Results from the stain-free method are in good agreement with traditional stain-based cytometric viability measurements. Additionally, when trained with images of healthy cells, dead cells and cells undergoing chemically induced apoptosis, the supervised fingerprinting algorithm is able to distinguish between the three cell states, and the results are independent of specific treatments or signaling pathways. We then show that an unsupervised variational autoencoder (VAE) algorithm trained on the same images, but without human-generated labels, is able to distinguish between samples of healthy, dead and apoptotic cells along with cellular debris based on learned morphological features and without human input. With this, we demonstrate that VAEs are a powerful exploratory technique that can be used as a process monitoring analytical tool.

Modeling scalability of impurity precipitation in downstream biomanufacturing.

Precipitation during the viral inactivation, neutralization and depth filtration step of a monoclonal antibody (mAb) purification process can provide quantifiable and potentially significant impurity reduction. However, robust commercial implementation of this unit operation is limited due to the lack of a representative scale-down model to characterize the removal of impurities. The objective of this work is to compare isoelectric impurity precipitation behavior for a monoclonal antibody product across scales, from benchtop to pilot manufacturing. Scaling parameters such as agitation and vessel geometry were investigated, with the precipitate amount and particle size distribution (PSD) characterized via turbidity and flow imaging microscopy. Qualitative analysis of the data shows that maintaining a consistent energy dissipation rate (EDR) could be used for approximate scaling of vessel geometry and agitator speeds in the absence of more detailed simulation. For a more rigorous approach, however, agitation was simulated via computational fluid dynamics (CFD) and these results were applied alongside a population balance model to simulate the trajectory of the size distribution of precipitate. CFD results were analyzed within a framework of a two-compartment mixing model comprising regions of high- and low-energy agitation, with material exchange between the two. Rate terms accounting for particle formation, growth and breakage within each region were defined, accounting for dependence on turbulence. This bifurcated model was successful in capturing the variability in particle sizes over time across scales. Such an approach enhances the mechanistic understanding of impurity precipitation and provides additional tools for model-assisted prediction for process scaling.

Features in Backgrounds of Microscopy Images Introduce Biases in Machine Learning Analyses

Subvisible particles may be encountered throughout the processing of therapeutic protein formulations. Flow imaging microscopy (FIM) and backgrounded membrane imaging (BMI) are techniques commonly used to record digital images of these particles, which may be analyzed to provide particle size distributions, concentrations, and identities. Although both techniques record digital images of particles within a sample, FIM analyzes particles suspended in flowing liquids, whereas BMI records images of dry particles after collection by filtration onto a membrane. This study compared the performance of convolutional neural networks (CNNs) in classifying images of subvisible particles recorded by both imaging techniques. Initially, CNNs trained on BMI images appeared to provide higher classification accuracies than those trained on FIM images. However, attribution analyses showed that classification predictions from CNNs trained on BMI images relied on features contributed by the membrane background, whereas predictions from CNNs trained on FIM features were based largely on features of the particles. Segmenting images to minimize the contributions from image backgrounds reduced the apparent accuracy of CNNs trained on BMI images but caused minimal reduction in the accuracy of CNNs trained on FIM images. Thus, the seemingly superior classification accuracy of CNNs trained on BMI images compared to FIM images was an artifact caused by subtle features in the backgrounds of BMI images. Our findings emphasize the importance of examining machine learning algorithms for image analysis with attribution methods to ensure the robustness of trained models and to mitigate potential influence of artifacts within training data sets.

Supervised and unsupervised machine learning approaches for monitoring subvisible particles within an aluminum-salt adjuvanted vaccine formulation.

Suspensions of protein antigens adsorbed to aluminum-salt adjuvants are used in many vaccines and require mixing during vial filling operations to prevent sedimentation. However, the mixing of vaccine formulations may generate undesirable particles that are difficult to detect against the background of suspended adjuvant particles. We simulated the mixing of a suspension containing a protein antigen adsorbed to an aluminum-salt adjuvant using a recirculating peristaltic pump and used flow imaging microscopy to record images of particles within the pumped suspensions. Supervised convolutional neural networks (CNNs) were used to analyze the images and create "fingerprints" of particle morphology distributions, allowing detection of new particles generated during pumping. These results were compared to those obtained from an unsupervised machine learning algorithm relying on variational autoencoders (VAEs) that were also used to detect new particles generated during pumping. Analyses of images conducted by applying both supervised CNNs and VAEs found that rates of generation of new particles were higher in aluminum-salt adjuvant suspensions containing protein antigen than placebo suspensions containing only adjuvant. Finally, front-face fluorescence measurements of the vaccine suspensions indicated changes in solvent exposure of tryptophan residues in the protein that occurred concomitantly with new particle generation during pumping.

Osmotic behavior of t cells determined by flow imaging microscopy

Osmotic behavior of t cells determined by flow imaging microscopy

Morphologically-Directed Raman Spectroscopy as an Analytical Method for Subvisible Particle Characterization in Therapeutic Protein Product Quality

Subvisible particles (SVPs) are a critical quality attribute of injectable therapeutic proteins (TPs) that needs to be controlled due to potential risks associated with drug product quality. The current compendial methods routinely used to analyze SVPs for lot release provide information on particle size and count. However, chemical identification of individual particles is also important to address root-cause analysis. Herein, we introduce Morphologically-Directed Raman Spectroscopy (MDRS) for SVP characterization of TPs. The following particles were used for method development: (1) polystyrene microspheres, a traditional standard used in industry; (2) photolithographic (SU-8); and (3) ethylene tetrafluoroethylene (ETFE) particles, candidate reference materials developed by NIST. In our study, MDRS rendered high-resolution images for the ETFE particles (> 90%) ranging from 19 to 100 μm in size, covering most of SVP range, and generated comparable morphology data to flow imaging microscopy. Our method was applied to characterize particles formed in stressed TPs and was able to chemically identify individual particles using Raman spectroscopy. MDRS was able to compare morphology and transparency properties of proteinaceous particles with reference materials. The data suggests MDRS may complement the current TPs SVP analysis system and product quality characterization workflow throughout development and commercial lifecycle.

Measuring the effectiveness of an integrated algae bloom, T&O, and cyanotoxin monitoring program

Abstract The City of Wichita Falls (CWF) Cypress Environmental Laboratory (CEL) established an integrated monitoring program in response to extreme taste and odor (T&O) events in 2016. The program uses field monitoring with sondes, sensory analysis to describe types of odors, water quality bench testing, algae/cyanobacteria identification and enumeration by flow-imaging microscopy, actinomycetes culturing, T&O detection by gas chromatography-mass spectrometry (GC-MS), cyanotoxin screening by quantitative polymerase chain reaction (qPCR), and cyanotoxin detection by fluorometry. To date, the CEL had detected 16 T&O events prior to finished tap being discharged to the distribution system. This has allowed utility staff to take proactive measures to mitigate events, which has virtually eliminated consumer T&O complaints. This article provides a brief history of T&O analysis, means to monitor and mitigate blooms, T&O compounds, and cyanotoxins, and measures the effectiveness of the CEL's monitoring program by evaluating each T&O event, and specifically geosmin.

Sub-Visible Particle Classification and Label Consistency Analysis for Flow-Imaging Microscopy Via Machine Learning Methods

Sub-visible particles can be a quality concern in pharmaceutical products, especially parenteral preparations. To quantify and characterize these particles, liquid samples may be passed through a flow-imaging microscopy instrument that also generates images of each detected particle. Machine learning techniques have increasingly been applied to this kind of data to detect changes in experimental conditions or classify specific types of particles, primarily focusing on silicone oil. That technique generally requires manual labeling of particle images by subject matter experts, a time-consuming and complex task.In this study, we created artificial datasets of silicone oil, protein particles, and glass particles that mimicked complex datasets of particles found in biopharmaceutical products. We used unsupervised learning techniques to effectively describe particle composition by sample. We then trained independent one-class classifiers to detect specific particle populations: silicone oil and glass particles. We also studied the consistency of the particle labels used to evaluate these models. Our results show that one-class classifiers are a reasonable choice for handling heterogeneous flow-imaging microscopy data and that unsupervised learning can aid in the labeling process. However, we found agreement among experts to be rather low, especially for smaller particles (< 8 µm for our Micro-Flow Imaging data). Given the fact that particle label confidence is not usually reported in the literature, we recommend more careful assessment of this topic in the future.

Water Quality Monitoring Using Particle Analysis

Key TakeawaysRegular monitoring of particle concentration in conjunction with turbidity analysis provides supplementary data that can provide important water quality information.Particle analysis using flow imaging microscopy (FIM) can help operators better understand various treatment processes and algae growth in source waters.Specific size ranges can be monitored using FIM for target pathogens such as Cryptosporidium and Giardia.

Optimization of Flow Imaging Microscopy Setting Using Spherical Beads with Optical Properties Similar to Those of Biopharmaceuticals

Flow imaging microscopy (FIM) is widely used to characterize biopharmaceutical subvisible particles (SVPs). The segmentation threshold, which defines the boundary between the particle and the background based on pixel intensity, should be properly set for accurate SVP quantification. However, segmentation thresholds are often subjectively and empirically set, potentially leading to variations in measurements across instruments and operators. In the present study, we developed an objective method to optimize the FIM segmentation threshold using poly(methyl methacrylate) (PMMA) beads with a refractive index similar to that of biomolecules. Among several candidate particles that were evaluated, 2.5-µm PMMA beads were the most reliable in size and number, suggesting that the PMMA bead size analyzed by FIM could objectively be used to determine the segmentation threshold for SVP measurements. The PMMA bead concentrations measured by FIM were highly consistent with the indicative concentrations, whereas the PMMA bead size analyzed by FIM decreased with increasing segmentation threshold. The optimal segmentation threshold where the analyzed size was closest to the indicative size differed between an instrument with a black-and-white camera and that with a color camera. Inter-instrument differences in SVP concentrations in acid-stressed recombinant adeno-associated virus (AAV) and protein aggregates were successfully minimized by setting an optimized segmentation threshold specific to the instrument. These results reveal that PMMA beads can aid in determining a more appropriate segmentation threshold to evaluate biopharmaceutical SVPs using FIM.

Microplastics' Shape and Morphology Analysis in the Presence of Natural Organic Matter Using Flow Imaging Microscopy.

Ubiquitous microplastics in urban waters have raised substantial public concern due to their high chemical persistence, accumulative effects, and potential adverse effects on human health. Reliable and standardized methods are urgently needed for the identification and quantification of these emerging environmental pollutants in wastewater treatment plants (WWTPs). In this study, we introduce an innovative rapid approach that employs flow imaging microscopy (FlowCam) to simultaneously identify and quantify microplastics by capturing high-resolution digital images. Real-time image acquisition is followed by semi-automated classification using customized libraries for distinct polyethylene (PE) and polystyrene (PS) microplastics. Subsequently, these images are subjected to further analysis to extract precise morphological details of microplastics, providing insights into their behavior during transport and retention within WWTPs. Of particular significance, a systematic investigation was conducted to explore how the presence of natural organic matter (NOM) in WWTPs affects the accuracy of the FlowCam's measurement outputs for microplastics. It was observed that varying concentrations of NOM induced a more curled shape in microplastics, indicating the necessity of employing pre-treatment procedures to ensure accurate microplastic identification when utilizing the FlowCam. These observations offer valuable new perspectives and potential solutions for designing appropriate treatment technologies for removing microplastics within WWTPs.

Osmotic properties of T cells determined by flow imaging microscopy in comparison to electrical sensing zone analysis

To develop cryopreservation methods for cell-based medicinal products it is important to understand osmotic responses of cells upon immersion into solutions with cryoprotective agents (CPAs) and during freezing. The aim of this study was to assess the osmotic response of T cells by using flow imaging microscopy (FIM) as a novel cell-sizing technique, and to corroborate the findings with electrical impedance measurements conducted on a Coulter counter. Jurkat cells were used as a potential model cell line for primary T cells. Cell volume responses were used to derive important cell parameters for cryopreservation such as the osmotically inactive cell volume Vb and the membrane permeability towards water and various CPAs. Unlike Coulter counter measurement, FIM, combined with Trypan blue staining can differentiate between viable and dead cells, which yields a more accurate estimation of Vb. Membrane permeabilities to water, dimethyl sulfoxide (Me2SO) and glycerol were measured for Jurkat cells at different temperatures. The permeation of Me2SO into the cells was faster in comparison to glycerol. CPA permeation decreased with decreasing temperature following Arrhenius behavior. Moreover, membrane permeability to water decreased in the presence of CPAs. Vb of Jurkat cells was found to be 49% of the isotonic volume and comparable to that of primary T cells. FIM proved to be a valuable tool to determine the membrane permeability parameters of mammalian cells to water and cryoprotective agents, which in turn can be used to rationally design CPA loading procedures for cryopreservation.

Inducing an LCST in hydrophilic polysaccharides via engineered macromolecular hydrophobicity

Thermoresponsive polysaccharide-based materials with tunable transition temperatures regulating phase-separated microdomains offer substantial opportunities in tissue engineering and biomedical applications. To develop novel synthetic thermoresponsive polysaccharides, we employed versatile chemical routes to attach hydrophobic adducts to the backbone of hydrophilic dextran and gradually increased the hydrophobicity of the dextran chains to engineer phase separation. Conjugating methacrylate moieties to the dextran backbone yielded a continuous increase in macromolecular hydrophobicity that induced a reversible phase transition whose lower critical solution temperature can be modulated via variations in polysaccharide concentration, molecular weight, degree of methacrylation, ionic strength, surfactant, urea and Hofmeister salts. The phase separation is driven by increased hydrophobic interactions of methacrylate residues, where the addition of surfactant and urea disassociates hydrophobic interactions and eliminates phase transition. Morphological characterization of phase-separated dextran solutions via scanning electron and flow imaging microscopy revealed the formation of microdomains upon phase transition. These novel thermoresponsive dextrans exhibited promising cytocompatibility in cell culture where the phase transition exerted negligible effects on the attachment, spreading and proliferation of human dermal fibroblasts. Leveraging the conjugated methacrylate groups, we employed photo-initiated radical polymerization to generate phase-separated hydrogels with distinct microdomains. Our bottom-up approach to engineering macromolecular hydrophobicity of conventional hydrophilic, non-phase separating dextrans to induce robust phase transition and generate thermoresponsive phase-separated biomaterials will find applications in mechanobiology, tissue repair and regenerative medicine.

Biophysical characterization of siRNA-loaded lipid nanoparticles with different PEG content in an aqueous system

Although lipid nanoparticles (LNP) are potential carriers of various pharmaceutical ingredients, further investigation for maintaining their stability under various environmental stressors must be performed. This study evaluated the influence of PEGylation and stress conditions on the stability of siRNA-loaded LNPs with different concentrations of PEG (0.5 mol%; 0.5 % PEG-LNP and 1.0 mol%; 1.0 % PEG-LNP) anchored to their surface. We applied end-over-end agitation, elevated temperature, and repeated freeze and thaw (F/T) cycles as physicochemical stressors of pH and ionic strength. Dynamic light scattering (DLS), flow imaging microscopy (FIM), and ionic-exchange chromatography (IEX) were to determine the degree of aggregation and change in siRNA content. The results indicate that 0.5 % PEG-LNP resisted aggregation only at low pH levels or with salt, whereas 1.0 % PEG-LNP had increased colloidal stability except at pH 4. 0.5 % PEG-LNP withstood aggregation until 71 °C and three cycles of F/T. In contrast, 1.0 % PEG-LNP maintained colloidal stability at 90 °C and seven F/T cycles. Moreover, 1.0 % PEG-LNP had higher siRNA stability under all stress conditions. Therefore, to ensure the stability of LNP and encapsulated siRNA, the PEG concentration must be carefully controlled while considering LNPs’ colloidal instability mechanisms under various stress conditions.

Interfacial Adsorption Controls Particle Formation in Antibody Formulations Subjected to Extensional Flows and Hydrodynamic Shear

During their manufacturing and delivery to patients, therapeutic proteins are commonly exposed to various interfaces and to hydrodynamic shear forces. Although adsorption of proteins to solid–liquid interfaces is known to foster formation of protein aggregates and particles, the impact of shear remains controversial, in part because of experimental challenges in separating the effects of shear from those caused by simultaneous exposure to interfaces. Extensional flows (occurring when solutions flow through sudden contractions) exert localized elongational forces that have been suspected to be damaging to proteins. In this work, we measured aggregation and particle formation in formulations of polyclonal and monoclonal antibodies subjected to extensional flow, high shear (105 s−1) and exposure to stainless-steel/water interfaces. Modification of the surface charge at the stainless steel/water interface changed protein adsorption characteristics without altering shear profiles, enabling shear and interfacial interactions to be separated. Even under conditions where antibodies were subjected to high hydrodynamic shear and extensional flow, production of subvisible particles could be inhibited by modifying the stainless-steel surface charge to minimize antibody adsorption. Digital images of particles recorded by flow imaging microscopy (FIM) and analyzed with machine learning algorithms were consistent with a particle formation mechanism by which antibodies adsorb and aggregate at the stainless-steel/water interface and subsequently form particles when shear displaces the interfacial aggregates, transporting them into the bulk solution. Topographical differences measured using atomic force microscopy (AFM) supported the proposed mechanism by showing reduced levels of protein adsorption on surface-charge-modified stainless-steel.

Image classification of degraded polysorbate, protein and silicone oil sub-visible particles detected by flow-imaging microscopy in biopharmaceuticals using a convolutional neural network model

Degradation of polysorbates in biopharmaceutical formulations can induce the formation of sub-visible particles (SvPs) in the form of free-fatty acids (FFAs) and potentially protein aggregates. Flow-imaging microscopy (FIM) is one of the most common techniques for enumerating and characterizing the SvPs, allowing for collection of image data of the SvPs in the size ranges of two to several hundred micrometers. The vast amounts of data obtained with FIM do not allow for rapid manual characterization by an experienced analyst and can be ambiguous. In this work, we present the application of a custom convolutional neural network (CNN) for classification of SvP images of FFAs, proteinaceous particles and silicon oil droplets, by FIM. The network was then used to predict the composition of artificially pooled test samples of unknown and labeled data with varying compositions. Minor misclassifications were observed between the FFAs and proteinaceous particles, considered tolerable for application to pharmaceutical development. The network is considered to be suitable for fast and robust classification of the most common SvPs found during FIM analysis.

WITHDRAWN: Novel machine learning models for flow imaging microscopy sub-visible particle classification in protein formulations

Understanding the particulate content of formulated drug products is essential for ensuring patient safety. In particular, it is critical to assess the presence of aggregated proteins or extraneous particles (e.g. fibres) that pose potential dangers. Additionally, it is useful to be able to distinguish non-proteinaceous particles, such as silicone oil droplets that commonly occur in formulations stored in pre-filled syringes. Standard particle counting methods (e.g. light obscuration) provide only total numbers of particles of a given size, but provide no mechanism for particle classification. Significant recent work has focused on the use of flow imaging microscopy to enable simultaneous classification and counting of particles using machine learning (ML) models including convolutional neural networks (CNN). In this paper we expand upon this theme by exploring techniques for achieving high prediction accuracy when the size of the labeled dataset used for model training is limited. We demonstrate that maximum performance can be achieved by combining multiple techniques such as data augmentation, transfer learning, and novel (to this field) models combining imaging and tabular data.

Ballast water management systems (BWMS) Type approval: A new application for flow imaging microscopy (FlowCAM)

Ballast water is a well-known vector for non-indigenous organisms’ introduction, causing harm to ecology, economy, and human health. Following International Maritime Organization (IMO) regulations, all vessels should use an approved ballast water management system (BWMS) to reduce the transfer of organisms. As a result, rapid and accurate methods are needed for effective ballast water management. In this paper, we examined the applicability of a flow imaging microscopy (FlowCAM) for detecting phytoplankton in land-based and shipboard BWMS tests. The results showed that FlowCAM failed to recognize and tended to underestimate the abundance of some species in comparison to a fluorescence microscope. However, the abundance of many other species or genera estimated by the two methods was similar. Generally speaking, although some discrepancies were observed, considering the efficiency of FlowCAM in capturing images and measuring the abundance of particles, FlowCAM may become a promising technology in BWMS Type Approval.

An exploratory study on the effect of mechanical stress on particle formation in monoclonal antibody infusions.

Monoclonal antibody infusions (mAb-i) are administered for the treatment of various diseases. They are often transported over long distances from the compounding site to the site of administration. However, transport studies are typically carried out with the original drug product but not with compounded mAb-i. To address this gap, the impact of mechanical stress on the formation of subvisible/nanoparticles in mAb-i was investigated by dynamic light scattering and flow imaging microscopy. Different mAb-i concentrations were subjected to vibrational orbital shaking and stored at 2-8°C up to 35 days. The screening revealed that pembrolizumab and bevacizumab infusions show the highest propensity for particle formation. Especially bevacizumab at low concentrations exhibited an increase in particle formation. Because of the unknown health risks associated with the long-term application of subvisible particles (SVPs)/nanoparticles in infusion bags, stability studies carried out in the frame of licensing application procedures should also focus on SVP formation in mAb-i. In general, pharmacists should minimize the time of storage and mechanical stress during transport, especially in the case of low-concentrated mAb-i. Moreover, if siliconized syringes are used, they should be washed once with saline solution to minimize particle entry.

Detection of ice core particles via deep neural networks

Abstract. Insoluble particles in ice cores record signatures of past climate parameters like vegetation dynamics, volcanic activity, and aridity. For some of them, the analytical detection relies on intensive bench microscopy investigation and requires dedicated sample preparation steps. Both are laborious, require in-depth knowledge, and often restrict sampling strategies. To help overcome these limitations, we present a framework based on flow imaging microscopy coupled to a deep neural network for autonomous image classification of ice core particles. We train the network to classify seven commonly found classes, namely mineral dust, felsic and mafic (basaltic) volcanic ash grains (tephra), three species of pollen (Corylus avellana, Quercus robur, Quercus suber), and contamination particles that may be introduced onto the ice core surface during core handling operations. The trained network achieves 96.8 % classification accuracy at test time. We present the system's potential and its limitations with respect to the detection of mineral dust, pollen grains, and tephra shards, using both controlled materials and real ice core samples. The methodology requires little sample material, is non-destructive, fully reproducible, and does not require any sample preparation procedures. The presented framework can bolster research in the field by cutting down processing time, supporting human-operated microscopy, and further unlocking the paleoclimate potential of ice core records by providing the opportunity to identify an array of ice core particles. Suggestions for an improved system to be deployed within a continuous flow analysis workflow are also presented.