Good Experimental Design Research Articles

Ensemble Docking as a Tool for the Rational Design of Peptidomimetic Staphylococcus aureus Sortase A Inhibitors.

Sortase A (SrtA) of Staphylococcus aureus has long been shown to be a relevant molecular target for antibacterial development. Moreover, the designed SrtA inhibitors act via the antivirulence mechanism, potentially causing less evolutional pressure and reduced antimicrobial resistance. However, no marketed drugs or even drug candidates have been reported until recently, despite numerous efforts in the field. SrtA has been shown to be a tough target for rational structure-based drug design (SBDD), which hampers the regular development of small-molecule inhibitors using the available arsenal of drug discovery tools. Recently, several oligopeptides resembling the sorting sequence LPxTG (Leu-Pro-Any-Thr-Gly) of the native substrates of SrtA were reported to be active in the micromolar range. Despite the good experimental design of those works, their molecular modeling parts are still not convincing enough to be used as a basis for a rational modification of peptidic inhibitors. In this work, we propose to use the ensemble docking approach, in which the relevant SrtA conformations are extracted from the molecular dynamics simulation of the LPRDA (Leu-Pro-Arg-Asp-Ala)-SrtA complex, to effectively represent the most significant and diverse target conformations. The developed protocol is shown to describe the known experimental data well and then is applied to a series of new peptidomimetic molecules resembling the active oligopeptide structures reported previously in order to prioritize structures from this work for further synthesis and activity testing. The proposed approach is compared to existing alternatives, and further directions for its development are outlined.

Half the price, twice the gain: How to simultaneously decrease animal numbers and increase precision with good experimental design.

Animal research often involves experiments in which the effect of several factors on a particular outcome is of scientific interest. Many researchers approach such experiments by varying just one factor at a time. As a consequence, they design and analyze the experiments based on a pairwise comparison between two groups. However, this approach uses unreasonably large numbers of animals and leads to severe limitations in terms of the research questions that can be answered. Factorial designs and analyses offer a more efficient way to perform and assess experiments with multiple factors of interest. We will illustrate the basic principles behind these designs, discussing a simple example with only two factors before suggesting how to design and analyze more complex experiments involving larger numbers of factors based on multiway analysis of variance.

Stop Fooling Yourself! (Diagnosing and Treating Confirmation Bias).

Confirmation bias (CB) is a cognitive bias that allows us to fool ourselves by selectively filtering data and distorting analyses to support favored beliefs or hypotheses. In this article, I will briefly review some classic experiments from cognitive psychology that illustrate what a powerful, pernicious, and insidious force CB is. I will then discuss how to recognize CB in our own thinking and behavior and describe specific elements of good experimental design that can mitigate its effects. These elements-such as randomization and blinding-are conceptually straightforward but often difficult in practice and therefore not as widely implemented as they should be.

Number of trials necessary to achieve a representative performance of accuracy and timing during combat shooting.

Acquisition evaluations are expensive, have a high time liability, and tend to prioritize engineering requirements over human factors and good experimental design. Shooting serials usually consist of static prone shooting to minimize movement variability, increase reliability of accuracy and timing data or use a single data point to make acquisition decisions. To better understand the number of trials required to achieve representative performance of accuracy and timing, 60 shots from the standing unsupported position while cyclically moving the weapon from the low ready to shoot was utilized. Intra-class correlations, standard error of measurement, minimal detectable change, and sequential averaging analysis (SAA) were used to evaluate the variables of radial error, shot interval, x-bias and y-bias over the 60 shots. The number of trials required to achieve an intraclass correlation of greater than 0.8 ranged from 2 (shot interval) to 58 (y-bias), whereas SAA varied between 3 (x-bias) and 43 (shot interval) trials. When averaging 10 shots at a time, the moving intraclass correlation remained above 0.8 for radial error and y-bias between 7 and 15 shots, shot interval from the second shot, but x-bias never reached 0.8. The number of trials required to satisfy each reliability method was inconsistent, in line with previous literature. Given the limitations identified in the literature as well as practical considerations such as the preference for prioritizing radial error reasonable performance stability can be achieved after 15 shots, and using the moving intraclass correlation results it is recommended that the first six shots are discarded with the following nine shots used for analysis.

Comparative assessment of plasma cortisol and fecal corticoid metabolites (FCM) of Atlantic salmon (Salmo salar L.) subjected to acute- and long-term stress

Farmed Atlantic salmon (Salmo salar L.) are subject to a variety of stressors throughout production. Elevations of cortisol level in the blood are one of the major endocrine primary stress responses in vertebrates and are widely used as stress indicator. However, blood sampling is invasive and stressful procedures. Cortisol instantly released into the blood at sampling can easily interfere with the initial stress response of interest. Fecal corticoid metabolites (FCM) have been suggested for a less invasive assessment of stress in fish. In the present study, we evaluated stress responses by using plasma cortisol and FCM as stress indicators in Atlantic salmon. Atlantic salmon (with average weight 42 g) were exposed to 3 different stressors: 1) parr-smolt transformation; 2) infestation with salmon lice (Lepeophtheirus salmonis) for two-weeks; 3) infection with infectious salmon anaemia virus (ISAV) for four weeks. The results demonstrated that FCM levels correlated well with the plasma cortisol levels at single-point sampling during long-term stress (p < 0.05). Significant increases of both plasma cortisol and FCM were found two weeks after initiated 24 h light. Each tank was sampled twice, with around 40-min interval. The chasing during the first sampling was used as an acute stressor. The effects of sampling procedures on plasma cortisol and FCM levels are compared. Blood and feces were analyzed by an Enzyme-Linked Immunosorbent Assay (ELISA). Plasma cortisol and FCM levels increased within 40 min after fish perceived an acute stressor when the average weight of fish was below 100 g. Attenuated cortisol stress responses to acute stress were found after fish experiencing long-term stress. Collectively, FCM reflects changes in plasma cortisol and can be an alternative to analysis of plasma cortisol in Atlantic salmon, complementing evidence of using FCM as stress indicators for monitoring and improving fish welfare. The results highlight that standard sampling procedures and good experimental designs should be established to ensure robust and repeatable results in the study of stress based on the FCM measurement. Future study will investigate representative timepoints for FCM sampling, with particular attention to the types of stress and sizes of the fish.

Why use a biobank? Sharing mutations, gene editing and navigating the COVID-19 pandemic.

Cryopreservation is seen as a key aspect of good colony management, which supports the drive towards improvements in animal care and the implementation of the 3Rs. However, following the advent of gene editing technologies, the generation of new rat and mouse models is quicker and cheaper than ever before. This has led some to question the future value of biobanks around the world. In the following commentary we argue that the need to cryopreserve rat and mouse strains and distribute them from well-funded repositories is as strong as it has ever been. Repositories should not be considered as simply collections of redundant model organisms. Biobanks have a vital role to play in good experimental design. They distribute identical, quality controlled mutant strains to the community and eliminate the need to recreate mice. Archived material provides a check point in the development of new strains of rodents that minimises genetic drift and breeding failures. Cryopreservation also makes resource sharing easier and cheaper, and improves animal care by eliminating the need for live animal shipments. Furthermore, routine cryopreservation of valuable strains protects them from unforeseen events, such as the SARS and COVID-19 pandemics, which were accompanied by the very real prospect of immediate lab closures and/or severe disruption to courier services, rendering live animal shipments non-viable.

General Additive Network Effect Models

In the interest of business innovation, social network companies often carry out experiments to test product changes and new ideas. In such experiments, users are typically assigned to one of two experimental conditions with some outcome of interest observed and compared. In this setting, the outcome of one user may be influenced by not only the condition to which they are assigned but also the conditions of other users via their network connections. This challenges classical experimental design and analysis methodologies and requires specialized methods. We introduce the general additive network effect (GANE) model, which encompasses many existing outcome models in the literature under a unified model-based framework. The model is both interpretable and flexible in modeling the treatment effect as well as the network influence. We show that (quasi) maximum likelihood estimators are consistent and asymptotically normal for a family of model specifications. Quantities of interest such as the global treatment effect are defined and expressed as functions of the GANE model parameters, and hence inference can be carried out using likelihood theory. We further propose the “power-degree” (POW-DEG) specification of the GANE model. The performance of POW-DEG and other specifications of the GANE model are investigated via simulations. Under model misspecification, the POW-DEG specification appears to work well. Finally, we study the characteristics of good experimental designs for the POW-DEG specification. We find that graph-cluster randomization and balanced designs are not necessarily optimal for precise estimation of the global treatment effect, indicating the need for alternative design strategies.

Physiological Effects and Human Health Benefits of Hibiscus sabdariffa: A Review of Clinical Trials.

Hibiscus sabdariffa Linn. Malvaceae (HS) is characterized by its edible calyxes. The HS calyxes are widely used for cosmetic, food, and medicinal applications. According to ethnobotanical evidence, decoction, infusion, or maceration extracts from HS calyxes have been used in folk medicine to treat many ailments. Moreover, several in vitro and in vivo studies have demonstrated the pharmacological properties and potential human health benefits of HS consumption. On the other hand, the evaluation of the physiological effects and health benefits of HS in clinical studies is most challenging. Therefore, this narrative review summarizes and discusses the physiological effects and health benefits of HS calyxes reported in clinical trials. Preparations obtained from HS calyxes (extracts, infusions, decoction, teas, beverages, capsules, and pills) are used as non-pharmacological therapies to prevent/control diverse chronic non-communicable diseases. The most-reported HS health benefits are its antihypertensive, antidyslipidemic, hypoglycemic, body fat mass reduction, nephroprotective, antianemic, antioxidant, anti-inflammatory, and anti-xerostomic activities; these effects are associated with the phytochemicals found in HS. Moreover, no adverse effects were reported during the clinical trials. However, clinical studies exhibited some limitations; thus, further studies are required to validate the clinical efficacy of HS in large-scale studies with higher doses and a good experimental design

Experimental Design in Two-Sided Platforms: An Analysis of Bias

We develop an analytical framework to study experimental design in two-sided marketplaces. Many of these experiments exhibit interference, where an intervention applied to one market participant influences the behavior of another participant. This interference leads to biased estimates of the treatment effect of the intervention. We develop a stochastic market model and associated mean field limit to capture dynamics in such experiments and use our model to investigate how the performance of different designs and estimators is affected by marketplace interference effects. Platforms typically use two common experimental designs: demand-side “customer” randomization ([Formula: see text]) and supply-side “listing” randomization ([Formula: see text]), along with their associated estimators. We show that good experimental design depends on market balance; in highly demand-constrained markets, [Formula: see text] is unbiased, whereas [Formula: see text] is biased; conversely, in highly supply-constrained markets, [Formula: see text] is unbiased, whereas [Formula: see text] is biased. We also introduce and study a novel experimental design based on two-sided randomization ([Formula: see text]) where both customers and listings are randomized to treatment and control. We show that appropriate choices of [Formula: see text] designs can be unbiased in both extremes of market balance while yielding relatively low bias in intermediate regimes of market balance. This paper was accepted by David Simchi-Levi, revenue management and market analytics.

Design Space by Design of Experiments

The quality of methods and products are usually influenced by several input factors. Research has recently focused on understanding the effects of multidimensional and interconnected input factors on the results of pharmaceutical products and analytical methods using Design of Experiment (DoE). Furthermore, it examines how DOE may be implemented, both for students and teachers, as well as highlighting historical perspectives on DOE. A good experimental design can help you make the most use of the available resources and make the analysis of the results easier. Collaborations between researchers and practitioners that are pushing the boundaries of experimental design are examined. It provides an overview of the principles and applications of the most common screening and response surface design, as well as creating mixtures designs.

How to design good experiments in marketing: Types, examples, and methods

In this article, we present key tenets of good experimental design and provide some practical considerations for industrial marketing researchers. We first discuss how experiments have the ability to assess causal claims. Next, we provide an experimental taxonomy table, which brings out the value and limitations of different types of experiments and maps the various goals of business marketing research within each category. Here, we pay particular attention to field experiments since they provide experimental realism by measuring respondents' actual behavior. We also provide a thorough discussion on important practical issues such as questions on experimental design, sample size, and how to involve business organizations in the implementing steps. The paper concludes by stressing the importance of combining data types (e.g., field plus laboratory experiments) and by offering methodological advice on how to analyze experimental data in marketing.

The effect of a transducer’s spatial averaging on an elastodynamic guided wave’s wavenumber spectrum

Elastodynamic guided waves propagate in an elastic solid which makes it difficult, if not impractical, to place a receiving transducer in the direct path of the propagating wave (as one would for an acoustic wave in a fluid medium). To account for this, receiving transducers are often placed on the surface of the solid waveguide such that the transducer surface is parallel to the wave propagation direction. This transducer orientation introduces spatial averaging, which causes the received signal to have an altered signal amplitude and mode bias. We investigate both of these effects and present a simple model from which we derive a scaling-ratio expression that describes the effects of spatial averaging. We then test its performance using finite-element simulations that incorporate “real-world” assumptions (e.g., transient waves, piezoelectric effects, etc.). The results from the simulations demonstrate that the scaling-ratio can characterize the effects resulting from spatial averaging. The scaling-ratio expression will be particularly useful when designing experiments involving high frequency (small wavelength) guided waves. Lastly, the proposed scaling-ratio expression could be applied to other sensing methods, like Laser Doppler Vibrometry (LDV) or piezoelectric waveguides with inter-digitated transducers, because of its generality. However, the authors are careful to note that the scaling-ratio expression is not intended as a replacement of multi-physics analysis or good experimental design, and the effects of spatial averaging should be avoided whenever possible.

Isothermal titration calorimetry (ITC): a standard operating procedure (SOP)

Isothermal titration calorimetry (ITC) is currently widely used in many applied areas of research, spanning protein-ligand binding, metal-ligand interactions, DNA/DNA or protein/DNA interactions, partition to membranes, and polymer surfactant interactions, to mention just a few. This is due to the availability of commercial instruments, and thus the production and spread of an accepted and widely followed SOP is felt by most users,in an effort to produce results that are scientifically correct and comparable. Therefore, within the efforts of Working Group 4 of the ARBRE-MOBIEU COST Action (CA15126), this ITC SOP was generated, alongside SOPs for several other biophysical techniques. Here, we discuss the factors that are fundamental for good experimental design and that need to be carefully considered, as well as machine calibration, in particular chemical calibration, linked to another outcome of Working Group 4 on ITC benchmarking, to be also published in this Special Issue.

ADOpy: a python package for adaptive design optimization.

Experimental design is fundamental to research, but formal methods to identify good designs are lacking. Advances in Bayesian statistics and machine learning offer algorithm-based ways to identify good experimental designs. Adaptive design optimization (ADO; Cavagnaro, Myung, Pitt, & Kujala, 2010; Myung, Cavagnaro, & Pitt, 2013) is one such method. It works by maximizing the informativeness and efficiency of data collection, thereby improving inference. ADO is a general-purpose method for conducting adaptive experiments on the fly and can lead to rapid accumulation of information about the phenomenon of interest with the fewest number of trials. The nontrivial technical skills required to use ADO have been a barrier to its wider adoption. To increase its accessibility to experimentalists at large, we introduce an open-source Python package, ADOpy, that implements ADO for optimizing experimental design. The package, available on GitHub, is written using high-level modular-based commands such that users do not have to understand the computational details of the ADO algorithm. In this paper, we first provide a tutorial introduction to ADOpy and ADO itself, and then illustrate its use in three walk-through examples: psychometric function estimation, delay discounting, and risky choice. Simulation data are also provided to demonstrate how ADO designs compare with other designs (random, staircase).

Focus on Data: Statistical Design of Experiments and Sample Size Selection Using Power Analysis.

PurposeTo provide information to visual scientists on how to optimally design experiments and how to select an appropriate sample size, which is often referred to as a power analysis.MethodsStatistical guidelines are provided outlining good principles of experimental design, including replication, randomization, blocking or grouping of subjects, multifactorial design, and sequential approach to experimentation. In addition, principles of power analysis for calculating required sample size are outlined for different experimental designs and examples are given for calculating power and factors influencing it.ResultsThe interaction between power, sample size and standardized effect size are shown. The following results are also provided: sample size increases with power, sample size increases with decreasing detectable difference, sample size increases proportionally to the variance, and two-sided tests, without preference as to whether the mean increases or decreases, require a larger sample size than one-sided tests.ConclusionsThis review outlines principles for good experimental design and methods for power analysis for typical sample size calculations that visual scientists encounter when designing experiments of normal and non-Gaussian sample distributions.

Swiss 3R Competence Centre: Promoting the 3Rs implementation

This contribution describes the history and activity of the Swiss 3R Competence Centre (3RCC), which has been founded in 2018. Among the most relevant activities, a call for project proposals on 3Rs was launched in 2018, with 47 proposals received for selection, followed by a call for nominations for a 3Rs award, which will be announced in Sept. 2019. Implementation of a 3Rs educational program has also been carried out since 2018. The 3RCC is also developing an e-Learning module to promote the implementation of alternatives to animal testing. Finally, the centre activity is addressed to help fostering good experimental design and biostatistics practices and support.

Examining the effect of lab instructions on students' critical thinking during a chemical inquiry practical

Developing students’ critical thinking skills is often seen as an important educational goal for inquiry assignments. In this study, we investigated to what extent pre-laboratory activities of a chemical inquiry assignment influence students' independent critical thinking. We compared two forms of pre-laboratory activities that are frequently used in educational practice to prepare students for their inquiry assignments: on the one hand paved road pre-laboratory activities that lead students with sensemaking preparatory questions and on the other, critical-thinking pre-laboratory activities in which students start with the development of an experiment plan using provided information and criteria for a good experimental design. We conducted this study two years in succession in senior year Dutch high school chemistry classes during an inquiry assignment that involved the study of the relation between reaction kinetics and molecular reaction mechanisms of organic nucleophilic substitution reactions (SN1/SN2). We focused on aspects associated with critical thinking, such as the desire to understand what is observed and to be able to adjust an existing method or model on the basis of experimental data. The results show that the design of pre-laboratory activities strongly influence the critical thinking exhibited by students during their inquiry activities, whereby students who perform critical thinking pre-laboratory activities are more motivated to think more deeply about the meaning of their measurements than students that perform paved road pre-laboratory activities.

CONSOLIDAÇÃO DE ROTINA ANALÍTICA PARA QUANTIFICAÇÃO DE FÓSFORO TOTAL EM CORPOS HÍDRICOS: Impactos para a Gestão dos Recursos Hídricos

Understanding the phosphorus dynamics in the ecosystem and its correct and accurate quantification is fundamental for proper decision making in water resources management. Thus, the aim of this study was to evaluate the best analytical method for quantifying phosphorus using three distinct acid digestion methodologies for total phosphorus extraction. The study areas were the Barigui and Passauna Rivers which have different land use and soil occupation, as well as water uses. The results showed that all proposed methods have good reproducibility with R² greater than 0.98, for low and high concentrations. However, in the samples quantification, method 2 presented the most satisfactory results, in which the smaller sample volume and reagents used, shorter digestion time and good precision and reproducibility of P concentration of the samples stand out. Improper use of an analytical procedure or calibration curve may underestimate the quantification of P in samples with low concentrations by about 70%, highlighting the importance of good experimental design and concern for the quality of the information generated.

The 1912 Douglas-Fir Heredity Study: Long-Term Effects of Climatic Transfer Distance on Growth and Survival

Abstract The 1912 Douglas-Fir Heredity Study is one of the first studies undertaken by the US Forest Service, and one of the first forest genetics studies in North America. The study considers provenance variation of 120 parent trees from 13 seed sources planted at five test sites in the Pacific Northwest. The unique, long-term nature of the study makes it valuable to revisit and consider its biological and historical significance. This analysis considers how far climatically Douglas-fir populations may be moved without incurring unacceptable declines in growth and survival. Results indicate that Douglas-fir seed sources may be moved at least 2° C cooler or warmer and still retain good long-term survival and productivity. However, projected future climate change beyond 2° C may lead to lower survival and productivity. One option to address these concerns is assisted migration; however, if seed sources are moved beyond 2–3° C to a cooler climate in anticipation of warming, or from a more continental to a maritime climate, we are likely to see increased mortality and associated losses in productivity in the near-term. Lessons from this study include: (1) pay attention to good experimental design; we were able to overcome limitations from the design by using new statistical approaches; (2) maladaptation may take time to develop; poorer survival was not evident until more than two decades after planting; and (3) long-term studies may have value for addressing new, unforeseen issues in the future.

Experimental Design and Sample Preparation in Forest Tree Metabolomics.

Appropriate experimental design and sample preparation are key steps in metabolomics experiments, highly influencing the biological interpretation of the results. The sample preparation workflow for plant metabolomics studies includes several steps before metabolite extraction and analysis. These include the optimization of laboratory procedures, which should be optimized for different plants and tissues. This is particularly the case for trees, whose tissues are complex matrices to work with due to the presence of several interferents, such as oleoresins, cellulose. A good experimental design, tree tissue harvest conditions, and sample preparation are crucial to ensure consistency and reproducibility of the metadata among datasets. In this review, we discuss the main challenges when setting up a forest tree metabolomics experiment for mass spectrometry (MS)-based analysis covering all technical aspects from the biological question formulation and experimental design to sample processing and metabolite extraction and data acquisition. We also highlight the importance of forest tree metadata standardization in metabolomics studies.