IDL-LTSOJ: Research and implementation of an intelligent online judge system utilizing DNN for defect localization

Abstract

The evolution of artificial intelligence has thrust the Online Judge (OJ) systems into the forefront of research, particularly within programming education, with a focus on enhancing performance and efficiency. Addressing the shortcomings of the current OJ systems in coarse defect localization granularity and heavy task scheduling architecture, this paper introduces an innovative Integrated Intelligent Defect Localization and Lightweight Task Scheduling Online Judge (IDL-LTSOJ) system. Firstly, to achieve token-level fine-grained defect localization, a Deep Fine-Grained Defect Localization (Deep-FGDL) deep neural network model is developed. By integrating Bidirectional Long Short-Term Memory (BiLSTM) and Bidirectional Gated Recurrent Unit (BiGRU), this model extracts fine-grained information from the abstract syntax tree (AST) of code, enabling more accurate defect localization. Subsequently, we propose a lightweight task scheduling architecture to tackle issues, such as limited concurrency in task evaluation and high equipment costs. This architecture integrates a Kafka messaging system with an optimized task distribution strategy to enable concurrent execution of evaluation tasks, substantially enhancing system evaluation efficiency. The experimental results demonstrate that the Deep-FGDL model improves the accuracy by 35.9% in the Top-20 rank compared to traditional machine learning benchmark methods for fine-grained defect localization tasks. Moreover, the lightweight task scheduling strategy notably reduces response time by nearly 6000ms when handling 120 task volumes, which represents a significant improvement in evaluation efficiency over centralized evaluation methods.