Cross-laboratory evaluation of multiplex bead assays including independent common reference standards for immunological monitoring of observational and interventional human studies.

Krista E Van Meijgaarden,Steven G Smith,Tom H M Ottenhoff,Anne M F H Drittij,Mei M Ho,Bhagwati Khatri,Roelof A De Paus,Jelle J Goeman,Simone A Joosten,Hazel M Dockrell,Helen Mcshane,Viswanathan V Krishnan

doi:10.1371/journal.pone.0201205

Abstract

BackgroundMultiplex assays are increasingly applied to analyze multicomponent signatures of human immune responses, including the dynamics of cytokine and chemokine production, in observational as well as interventional studies following treatment or vaccination. However, relatively limited information is available on the performance of the different available multiplex kits, and comparative evaluations addressing this important issue are lacking.Study designTo fill this knowledge gap we performed a technical comparison of multiplex bead assays from 4 manufacturers, each represented by 3 different lots, and with the assays performed by 3 different laboratories. To cross compare kits directly, spiked samples, biological samples and a newly made reference standard were included in all assays. Analyses were performed on 324 standard curves to allow for evaluation of the quality of the standard curves and the subsequent interpretation of biological specimens.ResultsManufacturer was the factor which contributed most to the observed variation whereas variation in lots, laboratory or type of detection reagent contributed minimally. Inclusion of a common reference standard allowed us to overcome observed differences in cytokine and chemokine levels between manufacturers.ConclusionsWe strongly recommend using multiplex assays from the same manufacturer within a single study and across studies that are likely to compare results in a quantitative manner. Incorporation of common reference standards, and application of the same analysis method in assays can overcome many analytical biases and thus could bridge comparison of independent immune profiling (e.g. vaccine immunogenicity) studies. With these recommendations taken into account, the multiplex bead assays performed as described here are useful tools in capturing complex human immune-signatures in observational and interventional studies.

Highlights

Multiplex bead arrays benefit from common reference standard necessarily represent a position of the Commission who will not be liable for the use made of such information
Incorporation of common reference standards, and application of the same analysis method in assays can overcome many analytical biases and could bridge comparison of independent immune profiling studies. With these recommendations taken into account, the multiplex bead assays performed as described here are useful tools in capturing complex human immune-signatures in observational and interventional studies
Multiplex bead assays are commonly used for monitoring of complex multicomponent signatures of human immune responses in observational as well as interventional studies, such as treatment or vaccination trials

Summary

Introduction

Multiplex bead assays are commonly used for monitoring of complex multicomponent signatures of human immune responses in observational as well as interventional studies, such as treatment or vaccination trials. Current commercial multiplex bead assays can be performed on as little as 25– 50 μl of culture supernatant from stimulated PBMC cultures, (diluted) venous blood or serum/ plasma, and enable analysis of a large number of analytes Another major advantage of these assays is that samples can be collected and stored, allowing serial measurements of samples from individuals in a single assay, thereby limiting inter-assay variation and optimising detection of possibly subtle perturbations in cytokines, chemokines and other secreted analytes over time [3, 4]. In the past few years many studies have been performed in which multiplex assays were tested for accuracy and reproducibility by including spiked samples or WHO standards, and compared to single analyte assays like ELISA and ELISpot [5,6,7,8] Their performance was evaluated in combination with optimized stimulation protocols for PBMC samples [9]. Relatively limited information is available on the performance of the different available multiplex kits, and comparative evaluations addressing this important issue are lacking.

Methods

Results

Discussion

Conclusion