Development of Adaptive Testing Method Based on Neurotechnologies

Abstract

Purpose of the study. The aim of the study is to create neural network models of modules in an adaptive testing system to design an individual testing trajectory.The research article discusses the implementation of an adaptive testing system in terms of introducing artificial neural network modules into its composition, which should solve the problem of choosing a topic and the complexity of the next question, taking into account previous answers and the complexity of previously asked questions, as well as the connectivity of topics and response time as a factor guessing or searching for an answer, thereby forming an individual testing trajectory.Materials and methods. In the course of the study, the data that affect the quality of the solution of the problem was analyzed, the general modular structure of the system was proposed, and the main data flows entering the input of an artificial neural network (ANN) were described. To solve the problem of choosing the complexity of a question, it is proposed to use a feed-forward network, a comparison of various ANN architectures and training parameters (weight update algorithms, loss functions, number of training epochs, packet sizes) is carried out. As an alternative, the possibility of using a recurrent ANN LSTM (Long-Short Term Memory) network is considered. All results were obtained using the high-level Keras library, which allows you to quickly start at the initial stages of research and get the first results. SGD, Adam, NAdam and RMSprop implemented in Keras were compared as optimizers to achieve faster convergence. Adam showed the best results in terms of accuracy, while the MSE loss function (mean square error) was used together with the optimizer. Traditionally, training was carried out for a large number of epochs; graphs of dependences of accuracy on the number of epochs for a different number of neurons in the hidden layer were experimentally obtained.Results. Based on the study, we can conclude that the obtained accuracy of the direct propagation network of 80-85% is quite sufficient for its use in the adaptive testing system. However, it remains to answer the question of the need to improve the efficiency of an already implemented network, and, therefore, to conduct research on methods to improve the efficiency of networks, including finer tuning of parameters and learning algorithms, as well as architecture.A well-known and obvious drawback of using LSTMs is their exactingness in terms of equipment and resources, both during training (the training process takes a significant amount of time) and during startup, in our case, it is supplemented by increased requirements for the training sample and casts doubt on the advisability of further study of LSTM networks when solving this task.Conclusion. The introduction of the proposed tools will allow implementing an adaptive testing system, with an intelligent selection of questions depending on the demonstrated level of knowledge of the test person to form an individual testing trajectory in order to determine the reliable level of knowledge of the test subject for the optimal number of questions asked.