Estimation Of Psychometric Functions Research Articles

Visual Field Estimation by Probabilistic Classification.

The gold standard clinical tool for evaluating visual dysfunction in cases of glaucoma and other disorders of vision remains the visual field or threshold perimetry exam. Administration of this exam has evolved over the years into a sophisticated, standardized, automated algorithm that relies heavily on specifics of disease processes particular to common retinal disorders. The purpose of this study is to evaluate the utility of a novel general estimator applied to visual field testing. A multidimensional psychometric function estimation tool was applied to visual field estimation. This tool is built on semiparametric probabilistic classification rather than multiple logistic regression. It combines the flexibility of nonparametric estimators and the efficiency of parametric estimators. Simulated visual fields were generated from human patients with a variety of diagnoses, and the errors between simulated ground truth and estimated visual fields were quantified. Error rates of the estimates were low, typically within 2dB units of ground truth on average. The greatest threshold errors appeared to be confined to the portions of the threshold function with the highest spatial frequencies. This method can accurately estimate a variety of visual field profiles with continuous threshold estimates, potentially using a relatively small number of stimuli.

ADOpy: a python package for adaptive design optimization.

Experimental design is fundamental to research, but formal methods to identify good designs are lacking. Advances in Bayesian statistics and machine learning offer algorithm-based ways to identify good experimental designs. Adaptive design optimization (ADO; Cavagnaro, Myung, Pitt, & Kujala, 2010; Myung, Cavagnaro, & Pitt, 2013) is one such method. It works by maximizing the informativeness and efficiency of data collection, thereby improving inference. ADO is a general-purpose method for conducting adaptive experiments on the fly and can lead to rapid accumulation of information about the phenomenon of interest with the fewest number of trials. The nontrivial technical skills required to use ADO have been a barrier to its wider adoption. To increase its accessibility to experimentalists at large, we introduce an open-source Python package, ADOpy, that implements ADO for optimizing experimental design. The package, available on GitHub, is written using high-level modular-based commands such that users do not have to understand the computational details of the ADO algorithm. In this paper, we first provide a tutorial introduction to ADOpy and ADO itself, and then illustrate its use in three walk-through examples: psychometric function estimation, delay discounting, and risky choice. Simulation data are also provided to demonstrate how ADO designs compare with other designs (random, staircase).

Online Machine Learning Audiometry.

A confluence of recent developments in cloud computing, real-time web audio and machine learning psychometric function estimation has made wide dissemination of sophisticated turn-key audiometric assessments possible. The authors have combined these capabilities into an online (i.e., web-based) pure-tone audiogram estimator intended to empower researchers and clinicians with advanced hearing tests without the need for custom programming or special hardware. The objective of this study was to assess the accuracy and reliability of this new online machine learning audiogram method relative to a commonly used hearing threshold estimation technique also implemented online for the first time in the same platform. The authors performed air conduction pure-tone audiometry on 21 participants between the ages of 19 and 79 years (mean 41, SD 21) exhibiting a wide range of hearing abilities. For each ear, two repetitions of online machine learning audiogram estimation and two repetitions of online modified Hughson-Westlake ascending-descending audiogram estimation were acquired by an audiologist using the online software tools. The estimated hearing thresholds of these two techniques were compared at standard audiogram frequencies (i.e., 0.25, 0.5, 1, 2, 4, 8 kHz). The two threshold estimation methods delivered very similar threshold estimates at standard audiogram frequencies. Specifically, the mean absolute difference between threshold estimates was 3.24 ± 5.15 dB. The mean absolute differences between repeated measurements of the online machine learning procedure and between repeated measurements of the Hughson-Westlake procedure were 2.85 ± 6.57 dB and 1.88 ± 3.56 dB, respectively. The machine learning method generated estimates of both threshold and spread (i.e., the inverse of psychometric slope) continuously across the entire frequency range tested from fewer samples on average than the modified Hughson-Westlake procedure required to estimate six discrete thresholds. Online machine learning audiogram estimation in its current form provides all the information of conventional threshold audiometry with similar accuracy and reliability in less time. More importantly, however, this method provides additional audiogram details not provided by other methods. This standardized platform can be readily extended to bone conduction, masking, spectrotemporal modulation, speech perception, etc., unifying audiometric testing into a single comprehensive procedure efficient enough to become part of the standard audiologic workup.

The Do's and Don'ts of Psychophysical Methods for Interpretability of Psychometric Functions and Their Descriptors.

Many areas of research require measuring psychometric functions or their descriptors (thresholds, slopes, etc.). Data for this purpose are collected with psychophysical methods of various types and justification for the interpretation of results arises from a model of performance grounded in signal detection theory. Decades of research have shown that psychophysical data display features that are incompatible with such framework, questioning the validity of interpretations obtained under it and revealing that psychophysical performance is more complex than this framework entertains. This paper describes the assumptions and formulation of the conventional framework for the two major classes of psychophysical methods (single- and dual-presentation methods) and presents various lines of empirical evidence that the framework is inconsistent with. An alternative framework is then described and shown to account for all the characteristics that the conventional framework regards as anomalies. This alternative process model explicitly separates the sensory, decisional, and response components of performance and represents them via parameters whose estimation characterizes the corresponding processes. Retrospective and prospective evidence of the validity of the alternative framework is also presented. A formal analysis also reveals that some psychophysical methods and response formats are unsuitable for separation of the three components of observed performance. Recommendations are thus given regarding practices that should be avoided and those that should be followed to ensure interpretability of the psychometric function, or descriptors (detection threshold, difference limen, point of subjective equality, etc.) obtained with shortcut methods that do not require estimation of psychometric functions.

Conjoint psychometric field estimation for bilateral audiometry.

Behavioral testing in perceptual or cognitive domains requires querying a subject multiple times in order to quantify his or her ability in the corresponding domain. These queries must be conducted sequentially, and any additional testing domains are also typically tested sequentially, such as with distinct tests comprising a test battery. As a result, existing behavioral tests are often lengthy and do not offer comprehensive evaluation. The use of active machine-learning kernel methods for behavioral assessment provides extremely flexible yet efficient estimation tools to more thoroughly investigate perceptual or cognitive processes without incurring the penalty of excessive testing time. Audiometry represents perhaps the simplest test case to demonstrate the utility of these techniques. In pure-tone audiometry, hearing is assessed in the two-dimensional input space of frequency and intensity, and the test is repeated for both ears. Although an individual's ears are not linked physiologically, they share many features in common that lead to correlations suitable for exploitation in testing. The bilateral audiogram estimates hearing thresholds in both ears simultaneously by conjoining their separate input domains into a single search space, which can be evaluated efficiently with modern machine-learning methods. The result is the introduction of the first conjoint psychometric function estimation procedure, which consistently delivers accurate results in significantly less time than sequential disjoint estimators.

Painfree and accurate Bayesian estimation of psychometric functions for (potentially) overdispersed data

The psychometric function describes how an experimental variable, such as stimulus strength, influences the behaviour of an observer. Estimation of psychometric functions from experimental data plays a central role in fields such as psychophysics, experimental psychology and in the behavioural neurosciences. Experimental data may exhibit substantial overdispersion, which may result from non-stationarity in the behaviour of observers. Here we extend the standard binomial model which is typically used for psychometric function estimation to a beta-binomial model. We show that the use of the beta-binomial model makes it possible to determine accurate credible intervals even in data which exhibit substantial overdispersion. This goes beyond classical measures for overdispersion—goodness-of-fit—which can detect overdispersion but provide no method to do correct inference for overdispersed data. We use Bayesian inference methods for estimating the posterior distribution of the parameters of the psychometric function. Unlike previous Bayesian psychometric inference methods our software implementation—psignifit 4—performs numerical integration of the posterior within automatically determined bounds. This avoids the use of Markov chain Monte Carlo (MCMC) methods typically requiring expert knowledge. Extensive numerical tests show the validity of the approach and we discuss implications of overdispersion for experimental design. A comprehensive MATLAB toolbox implementing the method is freely available; a python implementation providing the basic capabilities is also available.

Improving the Estimation of Psychometric Functions in 2AFC Discrimination Tasks

Ulrich and Vorberg (2009) presented a method that fits distinct functions for each order of presentation of standard and test stimuli in a two-alternative forced-choice (2AFC) discrimination task, which removes the contaminating influence of order effects from estimates of the difference limen. The two functions are fitted simultaneously under the constraint that their average evaluates to 0.5 when test and standard have the same magnitude, which was regarded as a general property of 2AFC tasks. This constraint implies that physical identity produces indistinguishability, which is valid when test and standard are identical except for magnitude along the dimension of comparison. However, indistinguishability does not occur at physical identity when test and standard differ on dimensions other than that along which they are compared (e.g., vertical and horizontal lines of the same length are not perceived to have the same length). In these cases, the method of Ulrich and Vorberg cannot be used. We propose a generalization of their method for use in such cases and illustrate it with data from a 2AFC experiment involving length discrimination of horizontal and vertical lines. The resultant data could be fitted with our generalization but not with the method of Ulrich and Vorberg. Further extensions of this method are discussed.

Estimation of psychometric functions using a hybrid up–down maximum likelihood procedure

Rapid, efficient, and reliable estimation of psychometric functions is a much sought after goal in psychophysics. Adaptive techniques of the up–down type have many advantages in this regard, but they are not without their limitations. An alternative approach is the use of maximum likelihood estimation in placing observations. Whereas the maximum likelihood method is as efficient if not more efficient than any other method of estimation, this property only holds for very large samples. The use of maximum likelihood estimation in adaptive testing has significant limitations when the sample size is very small. In this study, a hybrid adaptive procedure was developed using an up–down technique initially and then converting to a maximum likelihood procedure once sufficient samples have been obtained for reliable estimation using the latter technique. The up–down technique that is used reduces the step size systematically employing a rule analogous to that used in analog-to-digial converters and is referred to as the AD procedure. Monte Carlo simulations of experiments using the hybrid adaptive technique showed promising results in comparison with other adaptive procedures. [Research supported by Grant 5P50DC00178 from NIDCD.]

Application of confidence intervals and joint confidence regions to the estimation of psychometric functions

A mathematical description of a psychometric function with two free parameters is fit to fixed‐increment data by a nonlinear gradient search technique that incorporates a weighted least squares algorithm. The statistical confidence in the parameter estimates is considered by comparing results from standard confidence interval analysis with those from joint confidence region analysis. Confidence intervals estimate the variability of each parameter alone, ignoring the interaction between the parameters, whereas the joint confidence region gives the confidence in the joint estimation of both parameters together. In our study, both analyses were applied to fixed‐increment data from a variety of tests (interaural time and intensity discrimination and N0Sπ binaural detection) and subjects for two‐interval forced choice experiments. Results show that the parameter estimates are affected primarily by the data collected near the midpoint which is not surprising since the parameters in the model are the slope and translation at the midpoint. However, the joint confidence region analysis shows that the statistical confidence of the parameter estimates are greatly affected by data collected at levels that give performance near chance and near perfect. [Work supported by NIDCD (Grant DC00100).]

Estimation of psychometric functions from adaptive tracking procedures.

Because adaptive tracking procedures are designed to avoid stimulus levels far from a target threshold value, the psychometric function constructed from the trial-by-trial data in the track may be accurate near the target level but a poor reflection of performance at levels far removed from the target. A series of computer simulations was undertaken to assess the reliability and accuracy of psychometric functions generated from data collected in up-down adaptive tracking procedures. Estimates of psychometric function slopes were obtained from trial-by-trial data in simulated adaptive tracks and compared with the true characteristics of the functions used to generate the tracks. Simulations were carried out for three psychophysical procedures and two target performance levels, with tracks generated by psychometric functions with three different slopes. The functions reconstructed from the tracking data were, for the most part, accurate reflections of the true generating functions when at least 200 trials were included in the tracks. However, for 50- and 100-trial tracks, slope estimates were biased high for all simulated experimental conditions. Correction factors for slope estimates from these tracks are presented. There was no difference in the accuracy and reliability of slope estimation due to target level for the adaptive track, and only minor differences due to psychophysical procedure. It is recommended that, if both threshold and slope of psychometric functions are to be estimated from the trial-by-trial tracking data, at least 100 trials should be included in the tracks, and a three- or four-alternative forced-choice procedure should be used. However, good estimates can also be obtained using the two-alternative forced-choice procedure or less than 100 trials if appropriate corrections for bias are applied.

Loudness of Pure Tones at Low and Infrasonic Frequencies

Contours of equal loudness were determined in the frequency range 2–63 Hz and the loudness range 20–100 phon. The loudness curves run almost parallel in the infrasonic frequency range and much closer than in the audio region. Infrasound only a few dB above the hearing threshold will therefore seem loud and possibly annoying. The subjects were 20 normal hearing students aged between 18 and 25, and the psychometric method was based on maximum-likelihood estimation of psychometric functions.

Hybrid adaptive procedure for estimation of psychometric functions.

Adaptive psychophysical procedures that have been described in the literature generally fall into one of two categories. (1) Simple procedures, such as UDTR and PEST, can be implemented without an on-line computer. The decision to change testing level is based on the outcome of the few most recent trials, and the final estimate of threshold is given by the final testing level or the average of a few testing levels. (2) "Maximum likelihood" methods require an on-line computer. A parametric form of the psychometric function is assumed, and after each trial a maximum-likelihood estimate of the parameters of the psychometric function is made on the basis of all preceding trials. This estimate is used to set the next testing level and to estimate threshold. We describe here a hybrid procedure, in which testing levels are determined by PEST and the final estimate of threshold is made by fitting an assumed psychometric function to all preceding trials. The PEST rules were tuned to yield results that were accurate and insensitive to errors in initial estimates of the psychometric function. These parameters differ from those that yield optimum results with classical PEST. Results of computer simulations and of experiments with human subjects are presented.

A hybrid adaptive procedure for estimation of psychometric functions

Adaptive psychophysical procedures that have been described in the literature generally fall into one of two categories. (1) Simple procedures, such as UDTR and PEST, can be implemented without an on‐line computer. The decision to change testing level is based on the outcome of the few most recent trials, and the final estimate of threshold is given by the final testing level or the average of a few testing levels. (2) “Maximum likelihood” methods require an on‐line computer. A parametric form of the psychometric function is assumed, and after each trial a maximum‐likelihood estimate of the parameters of the psychometric function is made on the basis of all preceding trials. This estimate is used to set the next testing level and to estimate threshold. We describe here a hybrid procedure, in which testing levels are determined by a simple procedure, but the final estimate of threshold is made by fitting an assumed psychometric function to all preceding trials. This procedure yields accurate estimates of the parameters of the psychometric function, and it is reasonably insensitive to errors in initial estimates of these parameters. Results of computer simulations and of psychophysical experiments will be presented.