Building a Knowledge Verification System Using AI

Abstract

A persistent problem in database education is that conventional learning management systems and simple SQL autograders usually verify whether an answer is correct but do not diagnose the underlying subtopic-level misconception or provide transparent remediation guidance. This paper presents an intelligent SQL learning support system that integrates an educational ER schema, validator-based error typing, L1–L6 markup, explainable CART diagnostics, and a pilot multidimensional item-response modeling layer. The anonymized pilot dataset contained 60 students, 30 SQL/database tasks, and 2,105 labeled attempts. Expert annotation showed high agreement (Cohen's kappa = 0.833). Learning outcomes improved in both groups, but compared with the control group, the experimental group using adaptive AI feedback improved more strongly: +17.80 percentage points versus +3.79 percentage points. The gain difference was 14.01 percentage points (Welch t(47.30) = 13.86, p < 0.001, 95% CI [11.98; 16.05], Hedges g = 3.53). For error classification, CART achieved an accuracy of 0.779, macro-F1 = 0.689, weighted-F1 = 0.793, and log-loss = 0.618 on the held-out test set (n = 317). Random forest and gradient boosting produced higher predictive scores, but CART was retained as the primary diagnostic model because of its interpretability. The MIRT layer was calibrated on a 60 × 30 item-response matrix and a six-dimensional Q-matrix. The results support the feasibility of explainable SQL diagnostics while indicating that broader cross-institutional validation is required before generalization.

Keywords

Graphical Abstract

Novelty Statement

The contribution of this study is an explainable and statistically validated SQL system that links ER-schema-aware validators, expert-labelled error taxonomy, CART-based diagnostic rules, and a pilot MIRT/Q-matrix layer for adaptive feedback and competency modelling.