User learning can affect software system's response time

Abstract

A session-based software system provides various services to its end users through graphical user interfaces. A novice user of a service's user interface takes more think time—the average time to comprehend the content and its layout on the interface—in comparison to expert users. The think time gradually decreases as she repeatedly comprehends the same interface over time. This decrease in think time is the user learning phenomenon. Owing to this learning behavior, the proportion of users—at various learning levels for different services—changes dynamically leading to a difference in the workload. Traditionally though, workload specifications (required for system performance evaluation) never accounted for user learning behavior. They generally assumed a global mean think time, instead. In this work, we first report an experimental study that demonstrates the impact of user learning of the graphical user interfaces of session-based systems. We measure the performance data of a real system while it is getting used by synthetic users (who are) at various learning levels. Statistical tests on our measured data reveal that different human learning curves, combined with different amounts of users arriving the system with various learning levels for different services, produce different system performance. Next, we explain a queueing network (QN) performance model called CogQN that accounts for user learning. It is a multi-class QN model where each service and its learning level constitute a class of users for the service. We present the concept of learning-level dependent, class-switching probabilities to model the probabilistic transition of learning levels of users for different services. This novel concept enables us to avoid the exponential explosion of classes that would otherwise be suffered by traditional class-switching theory, had it been used in this work. We solve the model using discrete-event simulation and validate it against the empirical data. Our CogQN model captures the effect of user learning on overall mean response times across different learning conditions within 10% absolute error in comparison to empirical data.