Research Article | Open Access | CC Attribution Non-commercial | Published online: 16 March 2026 Credit Card Fraud Detection Using Hybrid XGBoost and Autoencoder Models

Tejas V. Gandhi,1 Pragati P. Gupta,1,* Chirag S. Gandhi,1 Sarvesh D. Gagare,1Vaishali Rajput,1 and Rohini Chavan,2

1 Department of Artificial Intelligence and Data Science, Vishwakarma Institute of Technology, Pune, 411037, Maharashtra, India

2 Department of Electronics and Telecommunication Engineering, Vishwakarma Institute of Technology, Pune, 411037, Maharashtra, India

*Email: pragati.gupta24@vit.edu ((P. P. Gupta))

J. Inf. Commun. Technol. Algorithms Syst. Appl., 2026, 2(1), 26303    https://doi.org/10.64189/ict.26303

Received: 15 January 2026; Revised: 12 February 2026; Accepted: 11 March 2026

Abstract

The rapid growth of digital payment systems and e-commerce platforms has significantly increased the volume of credit card transactions worldwide, consequently raising the risk of fraudulent financial activities. Detecting fraudulent transactions remains a challenging problem due to the highly imbalanced nature of transaction datasets and the constantly evolving behavior of fraud patterns. Traditional rule-based detection systems and single machine learning models often struggle to identify both known and previously unseen fraud patterns effectively. To address these challenges, this study proposes a hybrid credit card fraud detection framework that integrates supervised and unsupervised machine learning approaches. In the proposed framework, XGBoost is employed as the primary supervised learning model to identify known fraud patterns from labeled transaction data due to its strong performance in handling nonlinear feature interactions and class imbalance. To complement this approach, an autoencoder-based anomaly detection model is used to identify unusual transaction behaviors by analyzing reconstruction errors from normal transaction patterns. The outputs of these two models are combined using a logistic regression–based meta-classifier, which learns an adaptive fusion strategy for integrating supervised fraud probabilities and unsupervised anomaly scores. The proposed hybrid system was evaluated using the BankSim transaction dataset, which simulates realistic financial transaction behavior. Experimental results demonstrate that the standalone XGBoost model achieved an Average Precision (AP) of 0.79, an F1-score of 0.719, precision of 0.743, and recall of 0.697. The final hybrid meta-classifier model achieved an AP of 0.724, F1-score of 0.703, precision of 0.689, and recall of 0.718,indicating improved recall stability and robustness in detecting fraudulent transactions. These results highlight the potential of hybrid machine learning architectures for enhancing fraud detection reliability in financial systems.

Keywords:

Graphical Abstract

Novelty statement

This article discusses an empirical and systematic analysis of supervised, unsupervised, and meta-learning-based hybrid detection approaches to fraud detection, where practical trade-offs for data calibration for imbalanced data are presented.