Building a Knowledge Verification System Using AI

Abstract

A persistent problem in database education is that conventional learning management systems and simple SQL autograders usually verify whether an answer is correct but do not diagnose the underlying subtopic-level misconception or provide transparent remediation guidance. This paper presents an intelligent SQL learning support system that integrates an educational ER schema, validator-based error typing, L1–L6 markup, explainable CART diagnostics, and a pilot multidimensional item-response modeling layer. The anonymized pilot dataset contained 60 students, 30 SQL/database tasks, and 2,105 labeled attempts. Expert annotation showed high agreement (Cohen's kappa = 0.833). Learning outcomes improved in both groups, but compared with the control group, the experimental group using adaptive AI feedback improved more strongly: +17.80 percentage points versus +3.79 percentage points. The gain difference was 14.01 percentage points (Welch t(47.30) = 13.86, p < 0.001, 95% CI [11.98; 16.05], Hedges g = 3.53). For error classification, CART achieved an accuracy of 0.779, macro-F1 = 0.689, weighted-F1 = 0.793, and log-loss = 0.618 on the held-out test set (n = 317). Random forest and gradient boosting produced higher predictive scores, but CART was retained as the primary diagnostic model because of its interpretability. The MIRT layer was calibrated on a 60 × 30 item-response matrix and a six-dimensional Q-matrix. The results support the feasibility of explainable SQL diagnostics while indicating that broader cross-institutional validation is required before generalization.

Keywords

SQL educationAutomated assessmentExplainable feedbackCARTKnowledge tracingItem response theoryMultidimensional item response theoryAdaptive learning

Graphical Abstract

Novelty Statement

The contribution of this study is an explainable and statistically validated SQL system that links ER-schema-aware validators, expert-labelled error taxonomy, CART-based diagnostic rules, and a pilot MIRT/Q-matrix layer for adaptive feedback and competency modelling.