Web-based Experimentation Research Articles

Advanced Training in Web-Based Psychology Research

Abstract. This article describes Advanced Training Institutes (ATIs) and workshops on psychological experimentation conducted via the Internet. These programs, conducted since 2002, presented instruction that evolved over the decades to reflect changes in Web-based methods and techniques. The need for instruction in the methods and methodology of Web-based research has increased over the years, as more researchers adopted these methods without necessarily learning the methodological lessons developed from theory and experience. Links to instructional materials created for the workshops are provided. From events and trends that played out over time, including the story of methods that were once state-of-the-art and are no longer functional today, lessons hard-won from the past can be used to anticipate and plan future directions in Web-based experimentation.

Conducting online virtual environment experiments with uncompensated, unsupervised samples.

Web-based experimentation with uncompensated and unsupervised samples allows for a larger and more diverse sample population, more generalizable results, and faster theory to experiment cycle. Given that participants are unsupervised, it is still unknown whether the data collected in such settings would be of sufficiently high quality to support robust conclusions. Therefore, we investigated the feasibility of conducting such experiments online using virtual environment technologies. We conducted a conceptual replication of two prior experiments that have been conducted in virtual environments. Our results replicate findings previously obtained in conventional laboratory settings. These results hold across different device types of participants (ranging from desktop, through mobile devices to immersive virtual reality headsets), suggesting that experiments can be conducted online with uncompensated samples in virtual environments.

Complementing corpus analysis with web-based experimentation in research on World Englishes

Abstract Usage-based research in linguistics has to a large extent relied on corpus data. However, a feature’s “failure to appear in even a very large corpus (such as the Web) is not evidence for ungrammaticality, nor is appearance evidence for grammaticality” (Schütze and Sprouse 2013: 29). It is therefore advisable to complement corpus-based analyses with experimental data, so as to (ideally) obtain converging evidence. This paper reviews reasons for combining corpus linguistic with psycholinguistic experimental methods, and demonstrates how research on varieties of English can profit from experimentation. For a study of conversion in Asian Englishes, the maze task (Forster, Guerrera, and Elliot 2009; Forster 2010) was implemented with a web-based, open-source software. The results of the experiment dovetail with a previous analysis of the Corpus of Global Web-based English (Davies 2013). These results should encourage researchers not to base findings exclusively on corpus evidence, but corroborate them by means of experimental data.

Individual Differences in Uncertainty Tolerance Are not Associated With Cognitive Control Functions in the Flanker Task.

Cognitive control refers to the ability to make correct decisions concurrent to distracting information, and to adapt to conflicting stimulus configurations, eventually promoting goal-directed behavior. Previous research has linked individual differences in cognitive control to psychopathological conditions such as anxiety. However, a link with uncertainty tolerance (UT) has not been tested so far, although both constructs describe cognitive and behavioral performance in ambiguous situations, thus they share some similarities. We probed cognitive control in web-based experimentation (jsPsych) with a simple flanker task (N=111) and a version without confounds in episodic memory (N=116). Both experiments revealed two well-established behavioral indices: congruency effects (CEs) and congruency-sequence effects (CSEs). Only small-to-zero correlations emerged between CEs, UT, and need for cognitive closure (NCC), a personality trait inversely related to UT. A subtle correlation (r=.18) was noted in Experiment 2 between NCC and CSE. Throughout, Bayesian analyses provided anecdotal-to-moderate evidence for the null-hypotheses.

Promises and pitfalls of Web-based experimentation in the advance of replicable psychological science: A reply to Plant (2015).

In a recent letter, Plant (2015) reminded us that proper calibration of our laboratory experiments is important for the progress of psychological science. Therefore, carefully controlled laboratory studies are argued to be preferred over Web-based experimentation, in which timing is usually more imprecise. Here we argue that there are many situations in which the timing of Web-based experimentation is acceptable and that online experimentation provides a very useful and promising complementary toolbox to available lab-based approaches. We discuss examples in which stimulus calibration or calibration against response criteria is necessary and situations in which this is not critical. We also discuss how online labor markets, such as Amazon’s Mechanical Turk, allow researchers to acquire data in more diverse populations and to test theories along more psychological dimensions. Recent methodological advances that have produced more accurate browser-based stimulus presentation are also discussed. In our view, online experimentation is one of the most promising avenues to advance replicable psychological science in the near future.

Revising the diagnosis of congenital amusia with the Montreal Battery of Evaluation of Amusia.

This article presents a critical survey of the prevalent usage of the Montreal Battery of Evaluation of Amusia (MBEA; Peretz et al., 2003) to assess congenital amusia, a neuro-developmental disorder that has been claimed to be present in 4% of the population (Kalmus and Fry, 1980). It reviews and discusses the current usage of the MBEA in relation to cut-off scores, number of used subtests, manner of testing, and employed statistics, as these vary in the literature. Furthermore, data are presented from a large-scale experiment with 228 German undergraduate students who were assessed with the MBEA and a comprehensive questionnaire. This experiment tested the difference between scores that were obtained in a web-based study (at participants’ homes) and those obtained under laboratory conditions with a computerized version of the MBEA. In addition to traditional statistical procedures, the data were evaluated using Signal Detection Theory (SDT; Green and Swets, 1966), taking into consideration the individual’s ability to discriminate and their response bias. Results show that using SDT for scoring instead of proportion correct offers a bias-free and normally distributed measure of discrimination ability. It is also demonstrated that a diagnosis based on an average score leads to cases of misdiagnosis. The prevalence of congenital amusia is shown to depend highly on the statistical criterion that is applied as cut-off score and on the number of subtests that is considered for the diagnosis. In addition, three different subtypes of amusics were found in our sample. Lastly, significant differences between the web-based and the laboratory group were found, giving rise to questions about the validity of web-based experimentation.

Web-based system in mechatronics learning: Views of undergraduate engineering students

This work concerns a Web-based system for learning some principles and methods in a mechatronics system that can be accessed remotely over the Internet, at any time and from any location. The investigation involves a case study to illustrate a manufacturing process and evaluate the remote experimental procedure. The main aim reported here is to determine students’ perceptions towards Web-based versus traditional experiments and identify any differences in final grade point averages. The experiment was performed at National Changhua University of Education, Taiwan, with an implementation period covering six semesters and a total of 226 students divided randomly into ‘traditional’ and ‘Web-based’ groups. The Web-based system helped students understand the concepts and master the technologies associated with Web-based mechatronic monitoring and control. Four out of six semesters surveyed recorded statistically significant differences between student perceptions of traditional and Web-based experimentation.

Hands-on experiences of undergraduate students in Automatics and Robotics using a virtual and remote laboratory

Automatics and Robotics subjects are always greatly improved when classroom teaching is supported by adequate laboratory courses and experiments following the ''learning by doing'' paradigm, which provides students a deep understanding of theoretical lessons. However, expensive equipment and limited time prevent teachers having sufficient educational platforms, and several low cost and flexible solutions have been developed to permit an effective teaching in Automatics and Robotics at a reasonable cost. Virtual and remote laboratories are inside this group of solutions as Web-based experimentation tools which have demonstrated the importance and effectiveness of hand-on experiences. This paper presents an experience teaching based on a blended-learning method using as experimentation tool a virtual and remote robotic laboratory called RobUALab, which is also described in the paper, in Automatics and Robotics subjects of the Computer Science degree at the University of Alicante. Students experiment with a set of hand-on exercises about Automatics and Robotics using RobUALab, firstly in face-to-face classes where they experiment in-situ with the real plant and, afterwards, they access to the experimentation environment in order to finish remotely their practical exercises outside the laboratory. The results obtained in the evaluation of the educational methodology proposed attest its efficiency in terms of learning degree and performance of the students.

A Systematic Two-Layer Approach to Develop Web-Based Experimentation Environments for Control Engineering Education

This paper introduces the systematic approach currently used by the Department of Computer Science and Automatic Control of the Spanish University of Distance Learning (UNED) to develop Web-based laboratories for distance learning of topics with high technical contents such as control engineering. This approach differentiates two layers in the construction of web- based laboratories: the experimentation layer and the e-learning layer. For the experimentation layer, LabVIEW and data acquisition boards from National Instruments are used to create the server-side applications and Easy Java Simulations for the client-side interfaces. For the e-learning layer, the eMersion environment is used to support the required flexible educational scheme. This paper describes the programming techniques implemented and the design considerations that justify these particular choices. As an llustrative case-study, an example of development of a web-based application is discussed, in which an electrical drive servo-motor is introduced as a convenient setup to practice with motion control applications. Other significant examples of web-based experiments developed by the authors are also reported.

Development of a Web-Based Control Laboratory for Automation Technicians: The Three-Tank System

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> This paper introduces a Web-based control laboratory for experimentation on a nonlinear multiple-input–multiple-output (MIMO) system: the three-tank plant. Using this application, automation technician students can learn in a practical way many fundamental aspects of control processes. The client-side of the virtual laboratory has been developed using the programming support provided by <emphasis emphasistype="italic">Easy Java Simulations</emphasis> (EJS), an open-source tool for teachers with limited programming skills who want to create Java applications and applets. The lab has been integrated in the collaborative Web-based experimentation environment <emphasis emphasistype="italic">eMersion</emphasis> which provides more flexibility for the students performing the laboratory experiments. </para>

Collaborative Web-Based Experimentation in Flexible Engineering Education

The Ecole Polytechnique Federale de Lausanne (EPFL), Lausanne, Switzerland, is deploying a flexible learning scheme for selected pilot courses in engineering education. In such a scheme, traditional lectures and written exercises are combined with additional Web-based learning resources. The main objective of this initiative is to sustain the evolution from traditional teaching to active learning and to better integrate the increasing number of educational resources available online. In engineering education, a key activity to sustain the learning process is hands-on experimentation carried out using either simulation tools or real equipment. This paper describes how a collaborative Web-based experimentation environment has been introduced at the EPFL for providing more flexibility to students performing laboratory experiments in automatic control, biomechanics, and fluid mechanics. It particularly describes the eJournal, a Web service integrated in the proposed learning environment that enables the collection and sharing of preparatory notes and experimental results with both peers and teaching assistants.

From the editors

Welcome to the third issue of Volume 36. This issue provides five insightful manscripts covering a range of topics and methodologies. The first paper, by Yao and Murphy, lays out in helpful detail the differences between total outsourcers and application service providers. Drawing upon insights from a particular ASP, the paper develops a novel model of the transition in ASP vendor-client relationships and builds propositions about the state of the relationship as well as the transitioning of the relationship. This paper is what we would label a theory-building conceptual manuscript. Such papers are very useful in expanding our understanding of a particular topic that has not been the subject of heavy empirical research (unlike outsourcing, the more specific topic of ASP vendor-client relationships has not been the subject of extensive research). We hope to receive other theory-building conceptual paper submissions on various topics and publish them in future issues.The second paper in this issue, by Chung, Byrd, Lewis, and Ford, develops and tests a model of the relationship between IT infrastructure flexibility, mass customization and business performance. Using data collected from a sample of IS managers across a subset of Fortune 2000 firms, the study provides empirical evidence of the importance of IT infrastructure flexibility towards helping a firm achieve mass customization.The third paper, by Travica, uses a case study to lend evidence to a model on virtual organization and e-Commerce. Discussing virtual organization in terms of its assumptions, characteristics, and forms, the paper provides a holistic view of the various instanciations of virtualization in the context of organizations and then ties virtualization to the area of e-Commerce.This fourth paper, by Wade and Tingling, provides a comparison of web-based experimentation to lab-based experimentation. Drawing insights from a meta-analysis of the literature, the paper discusses the many scientific advantages of web experimentation, by which the authors refer not to web-based survey collection, but the actual manipulation of constructs offered in an experiment conducted over the web. This paper contributes to advances in the methodologies available to IS researchers.This issue closes with the paper by Serva, Benamati, and Fuller. Disbanding the myths, or perhaps merely the misguided understandings, of the constructs of trust and trustworthiness, the authors compare first, second, and third order effects of the constructs across three separate data sets dealing with trust and e-Commmerce. The authors provide a thoughtful analysis of the basis for determining the circumstances under which trustworthiness (ability, benevolence, and integrity) should be treated as a first or second order construct.We wish to thank our five editorial board members who served as Senior Editor on the papers in this issue: David Gefen, Sajda Qureshi, Detmar Straub, Leyland Pitt, and Antonis Stylianou. We greatly appreciate their service to Data Base! Finally, we would like to extend our sincere thanks to the following senior editors who have now completed their three year terms: Maryam Alavi, Leyland Pitt, Laurie Schatzberg, Al Segars, and Rick Watson. Collectively, they helped us greatly during the past three years and we appreciate having the opportunity to have worked with each of them.

The Electronic Laboratory Journal: A Collaborative and Cooperative Learning Environment for Web-Based Experimentation

Numerous tools have been developed for supporting the collaboration between students in education, tools that mainly include facilities for sharing documents and enabling discussions. However, these environments do not emphasize the use of facilities that sustain collaborative work in the framework of remote experimentation carried out by a group of students located at different places. The Electronic Laboratory Journal (eJournal) paradigm proposed in this paper is a collaborative and cooperative enviromnent for Web-based experimentation in engineering education. The eJournal enhances the traditional laboratory journal, by providing a group of students with Web-based tools to collect, annotate, organize and share the data chunks necessary to complete their experimentation assignments. The data chunks, called fragments, may be composed of numerous objects of any format, such as text, images, graphics, manuscripts, measurement logs or experimental results. Fragments can be uploaded from local disks or imported from Web components. The eJournal also handles the submission of results to the educators and facilitates remote supervision, assistance and tutoring of the students. The eJournal paradigm is currently assessed at the School of Engineering, the Ecole Polytechnique Federale de Lausanne (EPFL), in the framework of hands-on experimentation activities focusing on remote manipulation of real setups and Web-based simulation. This paper presents the eJournal environment, its application and its evaluation as an enabling Web-based application for flexible learning.