Experimental Design Issues Research Articles

The Past, Current and Future Research in Cerebellar TMS Evoked Responses-A Narrative Review.

Transcranial magnetic stimulation coupled with electroencephalography (TMS-EEG) is a novel technique to investigate cortical physiology in health and disease. The cerebellum has recently gained attention as a possible new hotspot in the field of TMS-EEG, with several reports published recently. However, EEG responses obtained by cerebellar stimulation vary considerably across the literature, possibly due to different experimental methods. Compared to conventional TMS-EEG, which involves stimulation of the cortex, cerebellar TMS-EEG presents some technical difficulties, including strong muscle twitches in the neck area and a loud TMS click when double-cone coils are used, resulting in contamination of responses by electromyographic activity and sensory potentials. Understanding technical difficulties and limitations is essential for the development of cerebellar TMS-EEG research. In this review, we summarize findings of cerebellar TMS-EEG studies, highlighting limitations in experimental design and potential issues that can result in discrepancies between experimental outcomes. Lastly, we propose a possible direction for academic and clinical research with cerebellar TMS-EEG.

Understanding the limits to animal cognition

The thriving field of comparative cognition examines the behaviour of diverse animals in cognitive terms. Comparative cognition research has primarily focused on the abilities of animals - what tasks they can do - rather than on the limits of their cognition - tasks that exceed an animal's cognitive abilities. We propose that understanding and identifying cognitive limits is as important as demonstrating the capacities of animal minds. Here, we identify challenges that have deterred the study of cognitive limits related to epistemic, practical and publication problems. The epistemic problem is concerned with how we can confidently infer a cognitive limit from null or negative results. The practical problem is how can we be certain our research has identified a cognitive limit rather than failures in tasks due to methodological or experimental design issues. The publication problem outlines the publication bias toward positive and exciting results over negative or null results in animal cognition. We propose solutions to these three challenges and examples of how to conduct research to confidently identify and confirm cognitive limits in animals. We believe a refocus on the cognitive limits of animals is the next step in the field of comparative cognition. Knowing the limits to the intelligence of different animals will aid us in appreciating the diversity of animal intelligence, and will resolve outstanding questions of how cognition evolves.

An Enhanced Quantitative Reverse Transcription-PCR Approach for Measuring Circulating MicroRNAs.

Short (22nt) RNAs called microRNAs (miRNAs) bind to and inhibit target messenger RNAs in gene regulatory networks. Recent study suggests that miRNAs circulate in a stable, cell-free form and that particular miRNAs in plasma or serum may be biomarkers for cancer and other diseases. Circulating miRNAs as biomarkers provide distinct challenges including pre-analytic variance and data standardization. We describe our qRT-PCR approach for measuring circulating miRNAs as biomarkers, as well as sample preparation, experimental design, and data processing issues.

Are the effects of servant leadership only spurious? The state of research on the causal effects of servant leadership, recommendations, and an illustrative experiment

Causality is essential in informing science and policy. In the present study, we investigate the current state of research regarding causality in the field of servant (and authentic) leadership and provide recommendations on how causally identified studies can be conducted. After explaining the methodological problems that potentially prevent causal inferences (i.e., endogeneity bias and issues in experimental design), we provide two systematic literature reviews of servant and authentic leadership showing that these problems remain very prevalent. We then discuss two solutions on how causal effects of servant leadership or perceptions thereof can be established: randomized experiments and instrumental variable regression. To illustrate our recommendations, we report an experiment on the effect of a combination of two servant leadership dimensions (i.e., stewardship and authenticity) on follower performance and also investigate the effect of combined stewardship and authenticity perceptions using instrumental variable regression. The results do not indicate that combined stewardship and authenticity behavior or perceptions affect follower performance. Our study can serve as a roadmap, especially for servant leadership researchers, to address potential endogeneity and conduct causally identified research.

EXPRESS: A case study of learning in plants: Lessons learned from pea plants.

To facilitate the study of learning in plants we share our experiences of trying to replicate Gagliano et al. (2016) pea plant experiment. In the course of our efforts, we identified eleven issues that must be addressed when attempting to replicate these experiments. The issues range from germination and transplantation of seedlings to experimental design and apparatus issues. We propose a number of solutions to overcome these hurdles.

A Review of the Use of Virtual Reality Studies in Social Psychology

Experience in virtual environments is becoming widespread in both daily life and scientific studies. With the advancement of virtual reality technology, new research questions can be posed. Thus, different experiences of the body, relationships with avatars, and the use of mobile and interactive media are increasingly preferred as research topics and research methods. In addition, the relationships that people establish in virtual environments have become a new type of interaction in social psychology that needs to be investigated. The aim of this review is to examine how virtual reality is handled both as a study tool and as a field of study in social psychology. First, technical terms and the necessary software and hardware are introduced, and the concepts of presence and immersion, which are important in the virtual reality literature, are explained. Subsequently, examples of studies conducted with virtual reality in areas such as the self, morality, identity, terror management, and social impact in the field of social psychology are presented. Technical limitations, such as the uncanny valley, virtual reality sickness, ethical issues, and experimental design issues that may arise in virtual reality studies, are discussed. Finally, the contributions of virtual reality to the field and what it can do are emphasized.

Effective design and inference for cell sorting and sequencing based massively parallel reporter assays

The ability to measure the phenotype of millions of different genetic designs using Massively Parallel Reporter Assays (MPRAs) has revolutionized our understanding of genotype-to-phenotype relationships and opened avenues for data-centric approaches to biological design. However, our knowledge of how best to design these costly experiments and the effect that our choices have on the quality of the data produced is lacking. In this article, we tackle the issues of data quality and experimental design by developing FORECAST, a Python package that supports the accurate simulation of cell-sorting and sequencing-based MPRAs and robust maximum likelihood-based inference of genetic design function from MPRA data. We use FORECAST's capabilities to reveal rules for MPRA experimental design that help ensure accurate genotype-to-phenotype links and show how the simulation of MPRA experiments can help us better understand the limits of prediction accuracy when this data are used for training deep learning-based classifiers. As the scale and scope of MPRAs grows, tools like FORECAST will help ensure we make informed decisions during their development and the most of the data produced. The FORECAST package is available at: https://gitlab.com/Pierre-Aurelien/forecast. Code for the deep learning analysis performed in this study is available at: https://gitlab.com/Pierre-Aurelien/rebeca.

Why the study of comparative psychology is important to neuroscientists.

The purpose of this contribution is threefold. First, is to acquaint neuroscientists with the area of psychology known as comparative psychology. Comparative psychology is the oldest of the organized social sciences with the term appearing as early as 1808. Many of the myriad issues of experimental design routinely faced by comparative psychologists are directly applicable to neuroscience. These issues include consistent definitions of psychological phenomena, the use of Morgan's canon to reduce unbridled anthropomorphism, and observation oriented modeling as a new statistical procedure to increase replication. Second, is a discussion of early comparative methods that may be of value to contemporary neuroscientists. Third, how the comparative approach can help the neuroscientist limit unfounded generalizations across species and develop more animal-friendly behavioral testing options tailored for the species or strain of interest. The articles closes with some recommendations on how comparative psychologists and neuroscientists can work more closely together.

Computational analysis of cancer genome sequencing data.

Distilling biologically meaningful information from cancer genome sequencing data requires comprehensive identification of somatic alterations using rigorous computational methods. As the amount and complexity of sequencing data have increased, so has the number of tools for analysing them. Here, we describe the main steps involved in the bioinformatic analysis of cancer genomes, review key algorithmic developments and highlight popular tools and emerging technologies. These tools include those that identify point mutations, copy number alterations, structural variations and mutational signatures in cancer genomes. We also discuss issues in experimental design, the strengths and limitations of sequencing modalities and methodological challenges for the future.

MZINBVA: variational approximation for multilevel zero-inflated negative-binomial models for association analysis in microbiome surveys.

As our understanding of the microbiome has expanded, so has the recognition of its critical role in human health and disease, thereby emphasizing the importance of testing whether microbes are associated with environmental factors or clinical outcomes. However, many of the fundamental challenges that concern microbiome surveys arise from statistical and experimental design issues, such as the sparse and overdispersed nature of microbiome count data and the complex correlation structure among samples. For example, in the human microbiome project (HMP) dataset, the repeated observations across time points (level 1) are nested within body sites (level 2), which are further nested within subjects (level 3). Therefore, there is a great need for the development of specialized and sophisticated statistical tests. In this paper, we propose multilevel zero-inflated negative-binomial models for association analysis in microbiome surveys. We develop a variational approximation method for maximum likelihood estimation and inference. It uses optimization, rather than sampling, to approximate the log-likelihood and compute parameter estimates, provides a robust estimate of the covariance of parameter estimates and constructs a Wald-type test statistic for association testing. We evaluate and demonstrate the performance of our method using extensive simulation studies and an application to the HMP dataset. We have developed an R package MZINBVA to implement the proposed method, which is available from the GitHub repository https://github.com/liudoubletian/MZINBVA.

Ranking and Selection with Covariates for Personalized Decision Making

We consider a problem of ranking and selection via simulation in the context of personalized decision making, in which the best alternative is not universal, but varies as a function of some observable covariates. The goal of ranking and selection with covariates (R&S-C) is to use simulation samples to obtain a selection policy that specifies the best alternative with a certain statistical guarantee for subsequent individuals upon observing their covariates. A linear model is proposed to capture the relationship between the mean performance of an alternative and the covariates. Under the indifference-zone formulation, we develop two-stage procedures for both homoscedastic and heteroscedastic simulation errors, respectively, and prove their statistical validity in terms of average probability of correct selection. We also generalize the well-known slippage configuration and prove that the generalized slippage configuration is the least favorable configuration for our procedures. Extensive numerical experiments are conducted to investigate the performance of the proposed procedures, the experimental design issue, and the robustness to the linearity assumption. Finally, we demonstrate the usefulness of R&S-C via a case study of selecting the best treatment regimen in the prevention of esophageal cancer. We find that by leveraging disease-related personal information, R&S-C can substantially improve patients’ expected quality-adjusted life years by providing a patient-specific treatment regimen.

Statistical exploratory analysis of mask-fill reproduction operators of Genetic Algorithms

To be successful, a search algorithm needs to balance exploration and exploitation. In Genetic Algorithms (GAs) this is achieved through proportionate selection of individuals and reproduction operators. GAs can suffer premature convergence, when the diversity of the population decreases over time and search gets trapped in a local optimum, returning a very poor quality solution. Mask-fill reproduction operators utilizes bit-masking oriented data structure (BMODS) that maintains a good ratio between exploration and exploitation. Mask-fill reproduction operators have been utilized in various applications, but the exploration and exploitation ability of mask-fill reproduction operators have not been compared against other reproduction operators. This paper describes a rigorous and practical statistical methodology for the exploratory analysis of the mask-fill reproduction operators. First, the issues of robust experimental design and setting the control parameters for implementing a GA is addressed. Second, the impact of various reproduction operator combinations are analysed. In this study, three crossover operators and five mutation operators are considered which creates fifteen crossover–mutation operator combinations. Third, the methodology is demonstrated by considering grammatical inference (GI) problem as domain of enquiry. A hybrid genetic algorithm integrated with Sequitur algorithm (GAWS) is proposed for GI. Numerical results are presented to describe the effect of crossover–mutation operator combinations. The performance of the proposed GAWS is compared against the state-of-the-art algorithms. Statistical test are conducted to determine the performance significance of both crossover–mutation operator combinations and the proposed GAWS algorithm.

Co-Registration of Eye Movements and Fixation—Related Potentials in Natural Reading: Practical Issues of Experimental Design and Data Analysis

A growing number of studies are using co-registration of eye movement (EM) and fixation-related potential (FRP) measures to investigate reading. However, the number of co-registration experiments remains small when compared to the number of studies in the literature conducted with EMs and event-related potentials (ERPs) alone. One reason for this is the complexity of the experimental design and data analyses. The present paper is designed to support researchers who might have expertise in conducting reading experiments with EM or ERP techniques and are wishing to take their first steps towards co-registration research. The objective of this paper is threefold. First, to provide an overview of the issues that such researchers would face. Second, to provide a critical overview of the methodological approaches available to date to deal with these issues. Third, to offer an example pipeline and a full set of scripts for data preprocessing that may be adopted and adapted for one’s own needs. The data preprocessing steps are based on EM data parsing via Data Viewer (SR Research), and the provided scripts are written in Matlab and R. Ultimately, with this paper we hope to encourage other researchers to run co-registration experiments to study reading and human cognition more generally.

Yogurt, cultured fermented milk, and health: a systematic review

Consumption of yogurt and other fermented products is associated with improved health outcomes. Although dairy consumption is included in most dietary guidelines, there have been few specific recommendations for yogurt and cultured dairy products. A qualitative systematic review was conducted to determine the effect of consumption of fermented milk products on gastrointestinal and cardiovascular health, cancer risk, weight management, diabetes and metabolic health, and bone density using PRISMA guidelines. English language papers in PubMed were searched, with no date restrictions. In total, 1057 abstracts were screened, of which 602 were excluded owing to lack of appropriate controls, potential biases, and experimental design issues. The remaining 455 papers were independently reviewed by both authors and 108 studies were included in the final review. The authors met regularly to concur, through consensus, on relevance, methods, findings, quality, and conclusions. The included studies were published between 1979 and 2017. From the 108 included studies, 76 reported a favorable outcome of fermented milks on health and 67 of these were considered to be positive or neutral quality according to the Academy of Nutrition and Dietetics’ Quality Criteria Checklist. Of the 32 remaining studies, the study outcomes were either not significant (28) or unfavorable (4), and most studies (18) were of neutral quality. A causal relationship exists between lactose digestion and tolerance and yogurt consumption, and consistent associations exist between fermented milk consumption and reduced risk of breast and colorectal cancer and type 2 diabetes, improved weight maintenance, and improved cardiovascular, bone, and gastrointestinal health. Further, an association exists between prostate cancer occurrence and dairy product consumption in general, with no difference between fermented and unfermented products. This article argues that yogurt and other fermented milk products provide favorable health outcomes beyond the milk from which these products are made and that consumption of these products should be encouraged as part of national dietary guidelines.Systematic review registration: PROSPERO registration no. CRD42017068953.

Experimental design issues in choice-based conjoint applied to patient choice in healthcare.

Choice-based conjoint (CBC) is used to understand how individuals develop preferences for decision alternatives. When decision alternatives can be described in terms of attributes, researchers want to determine the value respondents attach to various attribute levels. Popular in psychology, marketing, economics and other areas, CBC is now finding applications in healthcare to understand patient choice in healthcare policy, drug development, doctor-patient communications, etc. However, a lack of standard methodologies has served as a barrier to its use in healthcare. Therefore, there is a need to identify good research practices for CBC in healthcare. We review recent advances in CBC such as Pareto optimal choice sets, information per profile and reducing choice set sizes, as applied to patient choice.

Bayesian Optimization for Adaptive Experimental Design: A Review

Bayesian optimisation is a statistical method that efficiently models and optimises expensive “black-box” functions. This review considers the application of Bayesian optimisation to experimental design, in comparison to existing Design of Experiments (DOE) methods. Solutions are surveyed for a range of core issues in experimental design including: the incorporation of prior knowledge, high dimensional optimisation, constraints, batch evaluation, multiple objectives, multi-fidelity data, and mixed variable types.

Bayesian Design of Agricultural Disease Transmission Experiments for Individual Level Models

Abstract Here, we address the issue of experimental design for animal and crop disease transmission experiments, where the goal is to identify some characteristic of the underlying infectious disease system via a mechanistic disease transmission model. Design for such non-linear models is complicated by the fact that the optimal design depends upon the parameters of the model, so the problem is set in simulation-based, Bayesian framework using informative priors. This involves simulating the experiment over a given design repeatedly using parameter values drawn from the prior, calculating a Monte Carlo estimate of the utility function from those simulations for the given design, and then repeating this over the design space in order to find an optimal design or set of designs. Here we consider two agricultural scenarios. The first involves an experiment to characterize the effectiveness of a vaccine-based treatment on an animal disease in an in-barn setting. The design question of interest is on which days to make observations if we are limited to being able to observe the disease status of all animals on only two days. The second envisages a trial being carried out to estimate the spatio-temporal transmission dynamics of a crop disease. The design question considered here is how far apart to space the plants from each other to best capture those dynamics. In the in-barn animal experiment, we see that for the prior scenarios considered, observations taken very close to the beginning of the experiment tend to lead to designs with the highest values of our chosen utility functions. In the crop trial, we see that over the prior scenarios considered, spacing between plants is important for experimental performance, with plants being placed too close together being particularly deleterious to that performance.

Heterogenising study samples across testing time improves reproducibility of behavioural data

The ongoing debate on the reproducibility crisis in the life sciences highlights the need for a rethinking of current methodologies. Since the trend towards ever more standardised experiments is at risk of causing highly idiosyncratic results, an alternative approach has been suggested to improve the robustness of findings, particularly from animal experiments. This concept, referred to as “systematic heterogenisation”, postulates increased external validity and hence, improved reproducibility by introducing variation systematically into a single experiment. However, the implementation of this concept in practice requires the identification of suitable heterogenisation factors. Here we show that the time of day at which experiments are conducted has a significant impact on the reproducibility of behavioural differences between two mouse strains, C57BL/6J and DBA/2N. Specifically, we found remarkably varying strain effects on anxiety, exploration, and learning, depending on the testing time, i.e. morning, noon or afternoon. In a follow-up simulation approach, we demonstrate that the systematic inclusion of two different testing times significantly improved reproducibility between replicate experiments. Our results emphasise the potential of time as an effective and easy-to-handle heterogenisation factor for single-laboratory studies. Its systematic variation likely improves reproducibility of research findings and hence contributes to a fundamental issue of experimental design and conduct in laboratory animal science.

Small scale archimedes hydro power plant test station: Design and experimental investigation

Abstract: This paper deals with the experimental investigation and design issues of a small Archimedes screw turbine (AST) hydropower test station. Variations of the inclination angle, water flow ratio as well as rotation speed/revolutions per minute (rpm) were experimentally examined with respect to the turbine power output, torque and efficiency. Besides the experimental approach, the design approach was also addressed in detail. The main outcome of the experimental work was related to the optimum working regime investigation for the AST. Testing was obtained for three different inclination angles, 21°, 25° and 30°, and results were compared with the two types of generators; AC and DC generators. According to the experimental result, the maximal efficiency of the AST was found to be 64%, at an inclination angle of 21°, and with an associated water flow of 10 l/s. For the same previously specified conditions, the power output on the generator was about 35 W. It was also found that in order to achieve maximal power output on the generator (57 W), the AST requires an inclination angle of 30° and a water flow of 12.81 l/s. Based on the experimental findings, graphs were provided and addressed, related to the current, AC and DC generator voltages, AST losses and generator efficiency.

Assessment of Cognitive Phenotyping in Inbred, Genetically Modified Mice, and Transgenic Mouse Models of Alzheimer's Disease.

Genetically modified mouse models are being used predominantly to understand brain functions and diseases. Well-designed and controlled behavioral analyses of genetically modified mice have successfully led to the identification of gene functions, understanding of brain diseases, and development of treatments. Recently, complex and higher cognitive functions have been examined in mice with genetic mutations. Therefore, research strategies for cognitive phenotyping should be sophisticated and evolve to convey the exact meaning of the findings and provide robust translational tools for testing hypotheses and developing treatments. This review addresses issues of experimental design and discusses studies that have examined cognitive function using mouse strain differences, genetically modified mice, and transgenic mice for Alzheimer's disease.