Confidence Intervals for Individualized Performance Models

Abstract

IN THEIR LETTER “MOVING TOWARDS INDIVIDUALIZED PERFORMANCE MODELS,” REIFMAN ET AL.1 MAKE A NUMBER OF VALUABLE RECOMMENDATIONS FOR the development of procedures for individualized prediction of cognitive performance. They also comment on the first published approach for individualized performance modeling, which we developed to predict performance in the face of unknown traits and uncertain initial states in the homeostatic and circadian processes.2 Reifman and colleagues assert that “shorter-horizon predictions should be more accurate than longer ones,” claim that the way we estimate the confidence intervals is “not in line with Bayesian procedures,” and observe that “prediction uncertainty decreases with increasing horizons, which is nonsensical.”1 However, their belief that prediction uncertainty should always increase when predicting further into the future is incorrect. This can be demonstrated with a brief counterexample. Let's consider an individual with a daily sleep need of approximately 8 hours, whose sleep schedule is set to be fixed at 10 hours time in bed per night for, say, the next 7 days. We may not know the person's initial homeostatic state, and so performance predictions for the first waking period under the fixed sleep schedule may have large confidence intervals reflecting considerable prediction uncertainty. However, the physiology underlying sleep and wakefulness will cause the individual to dissipate any excess homeostatic pressure across the subsequent days with 10 hours time in bed. Thus, over time we can be increasingly confident that the individual will display little or no cognitive impairment, and our confidence intervals for this prediction should shrink accordingly. This asymptotic feature of sleep/wake regulation is embedded in the two-process model, through the saturating exponential build-up and decay curves of the homeostatic process.3 Since we employed the two-process model as the basis for our individualized modeling approach,2 our performance predictions may indeed show shrinking confidence intervals when predicting further into the future—in agreement with what is known about the physiology of sleep and wakefulness.