A consistency measure for psychometric measurements.
Adaptive tracking procedures in psychophysics may produce erroneous, "untypical" results and non-converging tracks due to, e.g., inattention of the test subject or external disturbances. This paper presents a multi-state psychometric model, which is used to rate the outcome of psychometric measurement procedures with a consistency measure. The consistency measure may be used for a post hoc, automated consistency estimation for any psychometric measurement procedure that can be modeled with a sigmoid psychometric function. The model calculates the log likelihood difference between single and two interleaved psychometric functions, potentially underlying a recorded adaptive track. A binary classifier was tested with a range of candidates for consistency measures with simulated, inconsistent tracks, and expert ratings of empirical tracks. The proposed consistency measure was identified as the best candidate to classify inconsistent tracks, while expert ratings were best predicted with the spectrum of the stimulus level, which is shown to be a suboptimal predictor of consistency. A threshold of the proposed measure for the German matrix sentence test is 10 to test for inconsistency, with a sensitivity of 60% and a specificity of 80%.
345
- 10.1109/mic.2013.20
- Mar 1, 2013
- IEEE Internet Computing
753
- 10.3758/bf03194543
- Nov 1, 2001
- Perception & Psychophysics
27228
- 10.1038/s41592-019-0686-2
- Feb 3, 2020
- Nature Methods
15
- 10.1177/2331216520970011
- Jan 1, 2020
- Trends in Hearing
28
- 10.1177/2331216520938929
- Jan 1, 2020
- Trends in Hearing
313
- 10.3109/00206090109073110
- Jan 1, 2001
- International Journal of Audiology
6
- 10.1121/10.0014898
- Oct 1, 2022
- The Journal of the Acoustical Society of America
1
- 10.1080/14992027.2022.2142683
- Nov 22, 2022
- International Journal of Audiology
61
- 10.1080/14992020500243851
- Jan 1, 2006
- International Journal of Audiology
58
- 10.3109/14992027.2015.1061708
- May 1, 2015
- International Journal of Audiology
- Research Article
80
- 10.3758/bf03212251
- May 1, 1992
- Perception & psychophysics
Because adaptive tracking procedures are designed to avoid stimulus levels far from a target threshold value, the psychometric function constructed from the trial-by-trial data in the track may be accurate near the target level but a poor reflection of performance at levels far removed from the target. A series of computer simulations was undertaken to assess the reliability and accuracy of psychometric functions generated from data collected in up-down adaptive tracking procedures. Estimates of psychometric function slopes were obtained from trial-by-trial data in simulated adaptive tracks and compared with the true characteristics of the functions used to generate the tracks. Simulations were carried out for three psychophysical procedures and two target performance levels, with tracks generated by psychometric functions with three different slopes. The functions reconstructed from the tracking data were, for the most part, accurate reflections of the true generating functions when at least 200 trials were included in the tracks. However, for 50- and 100-trial tracks, slope estimates were biased high for all simulated experimental conditions. Correction factors for slope estimates from these tracks are presented. There was no difference in the accuracy and reliability of slope estimation due to target level for the adaptive track, and only minor differences due to psychophysical procedure. It is recommended that, if both threshold and slope of psychometric functions are to be estimated from the trial-by-trial tracking data, at least 100 trials should be included in the tracks, and a three- or four-alternative forced-choice procedure should be used. However, good estimates can also be obtained using the two-alternative forced-choice procedure or less than 100 trials if appropriate corrections for bias are applied.
- Research Article
1
- 10.1121/1.4782439
- May 1, 2007
- The Journal of the Acoustical Society of America
Adaptive tracking procedures are widely used in psychophysics. The common, fixed step-size, up-down procedures are known to be biased and yield convergence dependent upon the step-size used [e.g., B. W. Edwards and G. H. Wakefield, J. Acoust. Soc. Am. 83, S17 (1988)], provide an estimate of only one parameter of the psychometric function, and in general make nonoptimal use of information (a priori and a posteriori) concerning a given subject’s psychometric function. Recently introduced Bayesian adaptive procedures [e.g., L. L. Kontsevich and C. W. Tyler, Vision Res. 39, 2729–2737 (1999)] make optimal use of all information in both the estimation of parameters of the psychometric function, as well as in stimulus placement via a minimum-entropy rule. No direct comparison of the up-down and Bayes procedures has been carried out, however. We present results of Monte Carlo simulations that demonstrate the Bayes procedure is as efficient, and often more efficient, than common up-down procedures, even when estimating more parameters than the up-down procedures. We will also present results from absolute-detection experiments with human subjects (with and without hearing loss) that demonstrate the Bayes procedures behavior outside the simulation environment. These experiments used a priori information derived from an audiogram database.
- Research Article
16
- 10.1121/1.390766
- Apr 1, 1984
- The Journal of the Acoustical Society of America
Previous studies have reported that rise time of sawtooth waveforms may be discriminated in either a categorical-like manner under some experimental conditions or according to Weber's law under other conditions. In the present experiments, rise time discrimination was examined with two experimental procedures: the traditional labeling and ABX tasks used in speech perception studies and an adaptive tracking procedure used in psychophysical studies. Rise time varied from 0 to 80 ms in 10-ms intervals for sawtooth signals of 1-s duration. Discrimination functions for subjects who simply discriminated the signals on any basis whatsoever as well as functions for subjects who practiced labeling the endpoint stimuli as " pluck " and "bow" before ABX discrimination were not categorical in the ABX task. In the adaptive tracking procedure, the Weber fraction obtained from the jnds of rise time was found to be a constant above 20-ms rise time. The results from the two discrimination paradigms were then compared by predicting a jnd for rise time from the ABX discrimination data by reference to the underlying psychometric function. Using this method of analysis, discrimination results from previous studies were shown to be quite similar to the discrimination results observed in this study. Taken together the results demonstrate clearly that rise time discrimination of sawtooth signals follows predictions derived from Weber's law.
- Research Article
3
- 10.1121/1.2019355
- Apr 1, 1982
- The Journal of the Acoustical Society of America
Previous studies have reported that rise time at the onset of a sawtooth waveform may be discriminated in a categorical‐like manner under some experimental conditions or according to Weber's law under other conditions. In the present experiments, rise time discrimination was examined with both the traditional ABX paradigm and an adaptive tracking procedure. Rise time varied from 0 to 80 ms in 10‐ms intervals at the onset of a one second duration sawtooth signal. Discrimination functions were not categorical in the ABX task. This result was true even for subjects who practiced labeling the endpoint stimuli as “pluck” and “bow” before discrimination. In the adaptive tracking procedure, the Weber fraction obtained from the JND's of rise time was constant above 20‐ms rise time. The results from the two paradigms were then compared by predicting a JND for rise time from ABX discrimination by means of the underlying psychometric function. Using this analysis, the apparently categorical discrimination results from previous studies were shown to be quite similar to the results observed in this study. Our results demonstrate clearly that rise time discrimination follows Weber's law. [Supported by NINCDS grant NS‐12179.]
- Research Article
1
- 10.2466/pms.1990.70.1.243
- Feb 1, 1990
- Perceptual and Motor Skills
PEST, an adaptive (tracking) procedure originally developed for sensory research, was modified for cognitive studies with the multiple sequential frames paradigm and a letter-search task. The program which controlled stimulus presentation was designed to track the frame duration required for successful search on 75% of trials (the duration threshold). For each of the three subjects used, the threshold increased as a linear function of the number of sequentially presented frames, which varied between 1 and 10. Each tracking run was immediately followed by 40 trials with frame-exposure time fixed at the computer-selected threshold. Performance on these fixed level trials was close to the expected 75% correct. The Discussion addresses issues related to the use of the threshold measure in basic and applied cognitive research, explores the present findings about performance on fixed level trials at the computer-selected threshold, and examines the increase in duration threshold with the number of sequentially presented frames.
- Research Article
1
- 10.1121/1.4783498
- Apr 1, 2009
- The Journal of the Acoustical Society of America
Increased temporal uncertainty might be expected to produce increased internal noise, resulting in poorer detection and shallower psychometric functions. An alternative model in the literature predicts steeper psychometric functions with increased uncertainty. As a test of these two alternatives, detection of a 20‐ms intensity increment was assessed for eight subjects in a two‐interval, forced‐choice adaptive procedure where the increment was presented 100, 150, 200, 250, or 300 ms after the onset of a 400‐ms, 4‐kHz tone presented at 70 dB SPL. In one condition, the temporal position varied randomly from trial to trial. In the other, the temporal position was fixed for an entire 50‐trial block. Detection thresholds were higher and psychometric functions obtained by analyzing the trial‐by‐trial data from the adaptive tracks were shallower in the higher temporal uncertainty condition. Thresholds in both conditions were higher for temporal positions more distant from the center of the 400‐ms tone. Results support a model in which temporal uncertainty contributes to internal noise resulting in shallower psychometric functions. [Work supported by R01 DC006648 and T32 DC000013.]
- Research Article
189
- 10.1016/j.neuron.2005.05.012
- Jun 1, 2005
- Neuron
Visibility Reflects Dynamic Changes of Effective Connectivity between V1 and Fusiform Cortex
- Research Article
- 10.1121/10.0037580
- Apr 1, 2025
- The Journal of the Acoustical Society of America
Adolescents continue to develop on some psychoacoustic tasks, but most studies use adaptive tracking procedures to estimate thresholds; therefore, the effect of prolonged maturation on psychometric functions is relatively unknown. Here, we tested adolescents (age 10–18) and young adults (age 19–23) on three conditions: temporal interval discrimination (100 ms standard), gap detection in bandpass noise, and frequency discrimination (1000 Hz standard). Performance was measured using the method of constant stimuli and a 3AFC task. Data were analyzed by measuring threshold (66.7% correct; proxy for sensitivity), the slope of the psychometric function (a proxy for internal noise), and the proportion of listeners who met criteria for outliers (more than 2 SD from the mean), had poorly fit psychometric functions, or were unable to complete the task. Preliminary results suggest that temporal interval discrimination thresholds, slopes, and proportion of excluded data did not change with age. Gap detection thresholds improved and slopes became steeper with development, and the proportion of excluded data decreased with increasing age. Frequency discrimination thresholds did not improve with age, but slopes became steeper and excluded data decreased with maturation. These data suggest that internal noise may be decreasing during adolescence, even when thresholds stay the same. [Funded by NIDCD.]
- Research Article
2
- 10.1121/1.419064
- May 1, 1997
- The Journal of the Acoustical Society of America
In separate experiments, detection and identification thresholds were obtained for a set of 25 common environmental sounds (e.g., dog barking, car starting). In the detection task, threshold values of the event-to-noise ratio (Ev/N) were established using an adaptive tracking procedure. The identification experiment was part of a larger individual differences study reported earlier [Watson et al., J. Acoust. Soc. Am. 99, 2516(A) (1996)]. A 3AFC identification task was used, with sounds presented at eight levels of Ev/N. Thresholds were estimated from the psychometric functions derived from the data. The correlation between thresholds in the two tasks indicates that the identifiability of these sounds is only weakly related to their detectability. Several models were evaluated to identify the properties that determine the detectability of environmental sounds. Leaky integrator and cross-correlation models were used to predict detection data; identification data were evaluated using a multidimensional scaling algorithm.
- Conference Article
1
- 10.1109/acoustics.2018.8502357
- Sep 1, 2018
In adaptive procedures used in psychoacoustics, the Wald sequential statistical test has been employed only in the PEST procedure. A specificity of the Wald test is that it allows selecting the target threshold value at any point on the psychometric function. This feature is useful when there are special test requirements, such as selecting a point on a psychometric function corresponding to a given detectability index value. Adaptive, staircase, up-down procedures are based on rules of signal level setting that are simple to implement but, in contrast to the Wald test, they allow to estimate the threshold level only for a few points on the psychometric function. In this study the feasibility of an adaptive, up-down staircase procedure which uses a decision rule based on the Wald sequential statistical test was examined. The staircase procedures, combined with the Wald test for selecting signal level, were tested as to their ability to converge at various threshold levels, their accuracy, and bias. The study was aimed to determine whether the Wald test may serve as a proper replacement for standard rules of signal level setting in adaptive up-down procedures.
- Research Article
23
- 10.1007/s13349-018-0306-3
- Sep 19, 2018
- Journal of Civil Structural Health Monitoring
Historical buildings demand constant surveying because anthropogenic (e.g., use, pollution or traffic vibration) and natural or environmental hazards (e.g., environmental changes or earthquakes) can endanger their existence and safety. Particularly, in the Andean region of South America, earthen historical constructions require special attention and investigation due to the high seismic hazard of the area next to the Pacific coast. Structural Health Monitoring (SHM) can provide useful, real-time information on the condition of these buildings. In SHM, the implementation of automatic tools for feature extraction of modal parameters is a crucial step. This paper proposes a methodology for the automatic identification of the structural modal parameters. An innovative and multi-stage approach for the automatic dynamic monitoring is presented. This approach uses the Data-Driven Stochastic Subspace Identification method complemented by hierarchical clustering for automatic detection of the modal parameters, as well as an adaptive modal tracking procedure for providing a clear visualization of long-term monitoring results. The proposed methodology is first validated in data acquired in an emblematic sixteenth century historical building: the monastery of Jeronimos in Portugal. After proving its efficiency, the algorithm is used to process almost 5000 events containing data acquired in the church of Andahuaylillas, a sixteenth century adobe building located in Cusco, Peru. The results in these cases demonstrate that accurate estimation of predominant modal parameters is possible in those complex structures even if relatively few sensors are installed.
- Research Article
26
- 10.1007/s12541-017-0022-7
- Feb 1, 2017
- International Journal of Precision Engineering and Manufacturing
The design of a flight control system for an under-actuated quadrotor aircraft in presence of parametric uncertainties and external disturbance is quite challenging. In this study, we propose an adaptive trajectory tracking control base on sliding mode approach, an adaptive command filtered backstepping technique to stabilize the attitude of the quadrotor and using online estimators to estimate unknown aerodynamic parameters and external disturbances. First of all, the mathematical model of a quadrotor unmanned aerial vehicle (UAV) is presented. The adaptive tracking trajectory position and attitude control scheme are then formulated and the perturbations in quadrotor system are compensated by employing special Lyapunov functions. Simulation results are given to demonstrate the validity and effectiveness of the proposed algorithm on a quadrotor model under different conditions.
- Research Article
1
- 10.1016/s1874-1029(08)60102-0
- Aug 1, 2009
- Acta Automatica Sinica
Adaptive Tracking and Disturbance Rejection for a Class of Distributed Systems
- Research Article
1
- 10.1121/1.2021322
- May 1, 1984
- The Journal of the Acoustical Society of America
A two‐interval forced‐choice adaptive tracking procedure was used to estimate the frequency difference limen (DL). In one condition, the frequency of the standard tone was constant for every trial with the frequency of the comparison tone varied according to the adaptive procedure. The interval between standard and comparison stimuli (ISI) was fixed. In contrast to this “fixed‐standard fixed‐ISI” condition, other conditions were tested in which the standard frequency and/or the ISI were chosen on each trial according to pseudo‐random algorithms. The results indicate that the frequency DL may increase by as much as a factor of 2 for some roving‐standard and random ISI conditions. In addition, a trial‐by‐trial record of each run was used to assess discrimination performance as a function of frequency and ISI. The decay of the sensory trace with increasing ISI, and uncertainty about signal frequency and timing appear to account for the changes in discrimination. [Supported by NIH.]
- Research Article
16
- 10.1121/1.415865
- Jul 1, 1996
- The Journal of the Acoustical Society of America
One class of adaptive psychophysical procedures was studied, using simulated and human observers. These procedures are those which require an increase in stimulus intensity after an incorrect response, and a decrease after k successive correct responses. This paper analyzes how step size and the value of k affect the mean and standard deviation of threshold estimates based on a k-down 1-up adaptive procedure. Computer simulations are used to study the bias in threshold estimates, which are most evident when larger step size and small values of k are used. The adaptive procedure can be characterized by a function called the imbalance of the track, the relative probability of adjusting the stimulus either up or down at equal stimulus distances from the equilibrium point. These imbalance functions can be used to understand the threshold biases obtained in the computer simulations. The computer simulations also show that the average number of reversals obtained per trial is dependent on different values of k, but are largely independent of step size. The standard error of the threshold estimates, however, varies systematically with step size, but are nearly independent of k. Finally, we compare the stability of threshold estimates for human listeners using two very different sets of parameters: a very large step size (approximately half the range of the psychometric function) with k = 4, and the conventional k = 3 with an initial 4-dB and a final 2-dB step size.
- New
- Research Article
- 10.1121/10.0039806
- Nov 1, 2025
- The Journal of the Acoustical Society of America
- New
- Research Article
- 10.1121/10.0039708
- Nov 1, 2025
- The Journal of the Acoustical Society of America
- New
- Research Article
- 10.1121/10.0039808
- Nov 1, 2025
- The Journal of the Acoustical Society of America
- New
- Research Article
- 10.1121/10.0039669
- Nov 1, 2025
- The Journal of the Acoustical Society of America
- New
- Research Article
- 10.1121/10.0039807
- Nov 1, 2025
- The Journal of the Acoustical Society of America
- New
- Research Article
- 10.1121/10.0039583
- Nov 1, 2025
- The Journal of the Acoustical Society of America
- Research Article
- 10.1121/10.0039665
- Oct 1, 2025
- The Journal of the Acoustical Society of America
- Research Article
- 10.1121/10.0039573
- Oct 1, 2025
- The Journal of the Acoustical Society of America
- Research Article
- 10.1121/10.0037228
- Oct 1, 2025
- The Journal of the Acoustical Society of America
- Addendum
- 10.1121/10.0039663
- Oct 1, 2025
- The Journal of the Acoustical Society of America
- Ask R Discovery
- Chat PDF
AI summaries and top papers from 250M+ research sources.