Empirical Validation Studies Research Articles

Elicited Imitation as a Measure of L2 Implicit Knowledge: An Empirical Validation Study

A key issue in the field of second language acquisition has been the difficulty of specifying accurate measures of implicit language knowledge. This paper describes the development of an elicited imitation test. Its design differs from that of most other elicited imitation tests in that it (a) requires test takers to focus attention first on the meaning of the utterance before repeating it and (b) some of the sentences that test takers are presented with are grammatical and others are ungrammatical. Test takers are asked to repeat sentences in correct English. It is hypothesized that (a) requiring test takers to respond to the meaning of an utterance reduces the likelihood that they will explicitly focus on linguistic form and thus access explicit language knowledge and that (b) spontaneous correction of incorrect sentences is a powerful indication of participants’ constraints on internal grammar (Munnich et al. 1994). The test is trialled on a baseline group of 20 native speakers and a sample of 95 second language learners. Evidence which would suggest that this test is a likely measure of implicit language knowledge is presented.

A New Look at Existential Psychotherapy

Existential psychotherapy has remained on the fringes of the mainstream practice of psychotherapy. One reason for its limited acceptance is that the literature has tended to be convoluted and existential psychotherapists' ideas heterogeneous. Another reason is the dearth of empirical validation studies. What if a more succinct, well-defined and research-friendly model of existential psychotherapy could be developed? An argument against such a manualized approach is that making the model more mechanized and structured goes against some of the tenets of existential psychotherapy. Another argument is that the heterogeneity of the field prohibits manualization of existential psychotherapy. Although these reservations have some legitimacy, the purpose of this paper is to develop a more clinically oriented adaptation of existential psychology and lay the groundwork for a manualized approach to existential psychotherapy.

Improving nursing home quality of care through outcomes data: the MDS quality indicators.

The quality of care provided to nursing home residents is a continuing source of concern throughout the world. In the United States, the Health Care Financing Administration mandated the use of a standardised resident assessment instrument, called the Minimum Data Set (MDS) which facilitated the development of a set of nursing home quality of care indicators. The MDS Quality Indicators were developed by a team of researchers at the University of Wisconsin-Madison and covered 12 domains, as well as associated risk factors. This initial set of indicators then went through an iterative process of review by national clinical panels, further empirical analysis, validation studies, and pilot tests, to confirm whether they were accurate, reliable and useful tools for identifying quality problems. The final set of MDS Quality Indicators includes 24 variables that include both processes and outcomes of care and prevalence and incidence measures. They are defined at both the resident and facility level. Four of the indicators are risk-adjusted. The indicators have several possible applications: a source of consumer information; to help guide policy makers; and to assist providers in conducting quality improvement initiatives and to evaluate the impact of these efforts.

Book review: Empirically Supported Therapies: Best Practice in Professional Psychology.

KEITH S. DOBSON and KENNETH D. CRAIG (Eds.) Empirically Supported Therapies: Best Practice in Professional Psychology Thousand Oaks, CA: Sage Publications, 1998, 333 pages (ISBN 0-7619-1076-X, US$36.95, Softcover) Reviewed by DAVID R. EVANS Since 1969, when the first Banff Conference was held, the focus has continued to be on treatment and its validity. The Banff Conference was conceived as a way to bring together leaders in behaviour therapy to present and discuss their evolving behavioural treatments and the empirical support for them. In many ways the focus of this book on empirically supported therapies comes full circle to remind us of the importance of the empirical support for the therapies we employ, and the original goals of the Banff Conference. The book is divided into two sections devoted to general theoretical issues on the one hand and empirically validated treatments in specific areas on the other. The lead chapter is by Paul Crits-Christoph, a member of the APA Division 12 task force. He discusses the task force report, some of the reactions to it, and some of the future concerns and possibilities. The chapter provides a good overview for the reader and an orientation for the rest of the book. Steven C. Hayes is the author of the second chapter devoted to a consideration of political, economic, and professional forces that argue for, and at times against, the adoption of scientific practice guidelines in psychological practice. This is a significant contribution to the book because it is important that all practicing psychologists understand the external forces demanding that we work toward empirical support for all of the treatments we employ. In the third chapter, Larry E. Beutler and Mary Baker address a number of methodological issues that must be considered in the design of research to provide empirical validation for a treatment, including therapy models and patient individual differences. This chapter captures the complexity required in future empirical research compared to many of our current and early ventures. Chapter 4 by John R. Weisz provides a good overview of empirical validation studies of treatments for children and adolescents, and raises three major concerns with current studies in the area. This contribution, although devoted to children and adolescents, alerts us to the importance of developmentally related variables such as context and age-specific condition when we are designing empirical validation studies. The last chapter in the first section is by Anna Beth Doyle, who draws our attention to culture, another major factor that must be considered as empirically supported treatments are identified. The author of this chapter claims that there are almost no empirical studies assessing the effectiveness of specific treatments with minority culture clients. This suggests an important area for future research in the development of culture-specific treatments and their empirical validation. The lead chapter in the second section of the book is authored by Peter D. McLean and Kent W. Anderson, and is focused on the evolution of empirically validated treatments for depression. The authors also discuss the interplay between factors related to various therapeutic orientations and empirical findings. This chapter provides a good overview of theory, research, and future directions in the treatment of depression. In Chapter 7, Andrew Christensen and Neil S. Jacobson discuss traditional behavioural couples therapy, their integrative behavioural couples therapy, and preliminary data comparing the two approaches. They conclude their chapter with a brief overview of some of their ideas about the difficulties of executing empirical studies of therapy, particularly couples therapy. …

Metrics applicable to software design

Applying metrics to software is a way to measure and improve software quality. Many metrics apply to software implementations (code), so they cannot be used early in the life cycle. We survey ten modularity and structural complexity metrics applicable to software designs, and summarize the results of empirical validation studies when they are available. We present a database schema from which most of these metrics can be computed. Aspects of each metric require further study and refinement. However, using some of them during design may still be beneficial.

V. Empirical validity of indirect mental health needs-assessment models in Colorado

This article presents the results of empirical validation studies of six statistical models currently available for indirectly estimating the need for alcohol, drug abuse, and mental health (ADM) services across a large geographic area such as a state. Initial analyses show that the basic assumptions regarding geographic variations in need which are shared by all such models can be substantiated by data from the Colorado Social Health Survey (CSHS) of diagnosable disorders, dysfunction in everyday living, and demoralization. Next, the performance of the original versions of the six models on three statistical tests of the match between model predictions and surveyed ADM service needs is presented. The tests show how well each model's predictions of need prevalence in 48 small subareas of Colorado compared with surveyed needs in those same areas. Results indicate that in their original form, none of the models were accurate predictors of surveyed need for ADM services in Colorado, but that they did show predictive potential in terms of their epidemiologic relationships with surveyed need. Following “optimization” of the model equations by adjusting their parameters to best fit the CSHS data, repeated analyses indicated that all models except one were considerably more accurate predictors than a flat-rate model based on the assumption of equal need across all subareas. A modified two-variable linear regression model developed by Stem was the best all-around performer on the evaluations of prediction accuracy, followed by the Synthetic Estimation model. Two “experimental” regression models containing the better predictors found among the original models (poverty and divorce rates) also performed very well. The demonstrated performance of these optimized and experimental models provides additional evidence of the utility of selected models as services planning and budgeting tool for states. These models are further evaluated with respect to frequently designated high-priority “target” populations in the next article.

Software sizing and costing models: a survey of empirical validation and comparison studies

Effective software cost and size estimation is essential for planning and resource allocation in software development. Recent empirical work which attempts to validate, or which compares, software size and cost estimation models and tools is surveyed. Some observations are made concerning the rather poor results frequently obtained and the need for calibration of models for specific development environments.

Software Sizing and Costing Models: A Survey of Empirical Validation and Comparison Studies

Effective software cost and size estimation is essential for planning and resource allocation in software development. Recent empirical work which attempts to validate, or which compares, software size and cost estimation models and tools is surveyed. Some observations are made concerning the rather poor results frequently obtained and the need for calibration of models for specific development environments.

Construção e validação de um teste de avaliação do desenvolvimento cognitivo para crianças de nível pré-escolar

Os autores apresentam neste estudo os resultados da validação preliminar, empírica e de constructo, de um novo teste grafo-perceptivo para avaliação do desenvolvimento cognitivo das crianças de nível pré-escolar. O teste denominado «O Robot» (autómato) consiste na cópia de um conjunto de figuras geométricas simples, estruturados sobre a forma de uma figura humana esquemática e simétrica. Utilizou-se uma amostra de 117 crianças distribuídas por quatro grupos: crianças normais, crianças com dificuldades de aprendizagem, crianças com Síndrome de Down e crianças autistas. Pressupõe-se que o desempenho no teste se relaciona diretamente com o desenvolvimento da coordenação visuomotora e da função gestaltica, globalmente avaliadora da chamada inteligência geral. O teste pretende ser um bom preditor do desempenho cognitivo no nível pré-escolar assim como do grau de prontidão das crianças para a aprendizagem da leitura e da escrita no âmbito da escolaridade formal.

Validation of building energy analysis simulation programs at the solar energy research institute

This article presents a synopsis of work conducted over a number of years at the Solar Energy Research Institute (SERI) related to the validation of building energy analysis simulation programs. The work described was supported by the U.S. Department of Energy, Solar Buildings and Conservation Divisions, and more recently has also been sponsored under Task VIII of the International Energy Agency. This overview includes a brief description of: • • the validation methodology developed at SERI; • • the monitoring techniques used for empirical validation studies; • • several intermodel comparisons; • • an analytical verification study; • • several empirical validation studies.

Simultaneous estimation of several percentiles

In many simulation studies the performance of the modeled system is characterized only by mean values of the performance indices. However, the percentiles are more informative in sum marizing the performance. Several simultaneous percentiles, pre sented for example in the form of the boxplot, provide the analyst with better understanding of the system's behaviour. The simultaneous estimation of several percentiles is a cumber some task. The extended P2-algorithm proposed in this study significantly reduces the computation of the estimation. The algorithm simultaneously estimates several percentiles without storing and sorting the observations. Empirical validation studies indicate the algorithm can be used in simulation studies.

Empirical validation of dynamic thermal computer models of buildings

A methodology for the validation of dynamic thermal models of buildings has been presented. The three techniques, analytical verification, inter-model comparisons and empirical validation have been described and their relative merits assessed by reference to previous validation work on ESP, SERIR'S, DEROB and BLAST. Previous empirical validation work on these models has been reviewed. This research has shown that numerous sources of error have existed in previous studies leading to uncertainty in model predictions. The effects of these errors has meant that none of the previous empirical validation studies would have produced conclusive evidence of internal errors in the models themselves. An approach towards developing tests to empirically validate dynamic thermal models is given.

MUSIC AND MEMORY: PREDICTORS FOR ATTAINED ESL ORAL PROFICIENCY

This article presents the results of an empirical concurrent validation study in which measures of musical ability (pitch, loudness, and rhythm), auditory discrimination, and memory were used to account for variance in attained ESL oral proficiency. Memory was a significant, though tenuous, predictor of variance in the dependent variable. In addition, significant differences were noted for the different language groups and for the different tests. The article concludes with a proposed research program to determine whether training in musical abilities might be a profitable adjunct to ESL instruction.

Interrogative suggestibility and delinquent boys: An empirical validation study

This study has several aims. First, to validate the Gudjonsson Suggestibility Scale (GSS) among delinquent and adolescent boys placed in an assessment and short-term treatment centre. Second, to test the hypothesis that adolescent boys are particularly susceptible to suggestions when they are ‘pressured’ by negative feedback and instructions. Finally, to investigate the hypothesis that suggestibility is related to memory recall and self-esteem. The GSS was administered to 31 boys (ages 11 to 16 yr) who had been independently rated by two teachers on measures of suggestibility and self-esteem. Suggestibility as measured by the GSS was found to correlate significantly with the teachers' ratings of suggestibility, supporting empirically the criterion-related validity of the GSS. Compared with young ‘normal’ adults the boys were no more likely to give in to suggestive questions than the adults, unless their performance was subjected to criticism and negative feedback. The results suggest that deliquent adolescents may be particularly responsive to interpersonal pressure during interrogation. The findings have important implications for police interrogation procedures.

Classifying Undecided Voters in Pre-Election Polls

The treatment of don't know, no opinion, or other nonsubstantive responses is a problem in many consumer research surveys. This paper looks at the problem in the context of 1980 presidential election opinion polls. During the campaign, a relatively large proportion of those interviewed said that they were undecided as to whom they planned to vote for. Discriminant analysis is used to allocate undecided voters to candidates. The method is validated by a postelection follow-up survey. Ian Fenwick is Associate Professor of Marketing at York University. Frederick Wiseman is the Patrick F. and Helen C. Walsh Research Professor in the College of Business Administration at Northeastern University. John F. Becker is President and James R. Heiman Vice-President of Becker Research Corporation, Boston. This research was supported in part by a grant from the Dean's Research Fund, College of Business Administration, Northeastern University. The authors express their appreciation to the referees for their helpful suggestions. An earlier version of this paper was presented at the annual conference of the Association of Consumer Research, October 1981. Public Opinion Quarterly Vol. 46:383-391 ? 1982 by the Trustees of Columbia University Published by Elsevier Science Publishing Co., Inc. 0033-362X/82/0046-383/$2.50 This content downloaded from 157.55.39.255 on Mon, 01 Aug 2016 05:45:06 UTC All use subject to http://about.jstor.org/terms 384 FENWICK, WISEMAN, BECKER, AND HEIMAN illustrated by applying the new technique to data collected by the Becker Institute from an October 1980 poll of registered voters in Massachusetts. Further evidence is provided by an empirical validation study using data obtained in a November 1980 postelection survey of a sample of previously undecided voters.

The Construction and Validation of a Questionnaire for Measuring Attitudes Towards Learning Foreign Languages

ABSTRACT A questionnaire was constructed for measuring the attitudes of schoolchildren towards learning French. Factor analysis revealed a primary factor accounting for over 60% of the total variance, and an empirical validation study showed that the attitudes of children who obtained high scores on teachers’ ratings of attentiveness and enthusiasm were significantly better than those given low ratings. A subsequent replication using a large sample of pupils provided a clarification of the factor structure, and the results of an independent study of attitudes towards learning German showed that the questionnaire can easily be adapted to measure attitudes towards learning other foreign languages as well as French.

Signal detection theory pain measures: Empirical validation studies and adaptation-level effects

Proponents of the use of signal detection theory (SDT) in the assessment of pain modulation have generally looked for changes in d′ to indicate a reduction of sensory function, and a change in criterion to indicate a modification of the subject's response bias or attitudinal predisposition. In the first experiment, both assumptions failed to receive empirical verification. Discrimination d′ was equivalent before and after two strong levels of electrical current were reduced. The criterion parameter appeared to shift in a more conservative direction after the stimulus diminution. These results are used to question the validity of both detection and discrimination indices in the measurement of pain. An alternative means for describing the experimental results revealed a striking adaptation-level effect with implications for the assessment of both experimentally induced and endogenous pain. The outcome of a second experiment reinforced the adaptation-level theory interpretation of the results and provided additional evidence concerning the difficulties in evaluating SDT parameters in studies of potential analgesics.

Weighted Ratings: Performance Criteria for Concurrent Valdations of a Police Selection Test

Where time constraints and other practical problems make empirical validation studies impossible, concurrent validation-using weighted ratings as performance criteria-is an alternative with a number of advantages.

Metrics Applicable To Software Design

Applying metrics to software is a way to measure and improve software quality. Many metrics apply to software implementations (code), so they cannot be used early in the life cycle. We survey ten modularity and structural complexity metrics applicable to software designs, and summarize the results of empirical validation studies when they are available. We present a database schema from which most of these metrics can be computed. Aspects of each metric require further study and refinement. However, using some of them during design may still be beneficial.