Comparative Analysis of XGBoost and TabNet Approaches For E-commerce Fraud Detection

Abstract

The rapid growth of e-commerce has increased transaction convenience but has also intensified the risk of fraudulent activities, particularly in highly imbalanced transaction datasets. Effective fraud detection systems are therefore essential to minimize financial losses and maintain consumer trust. This study aims to compare the performance of an ensemble learning approach, Extreme Gradient Boosting (XGBoost), and a deep neural network model designed for tabular data, TabNet, in detecting fraud in e-commerce transactions. The research employs a quantitative comparative method using a publicly available e commerce fraud detection dataset consisting of 299,695 transaction records. Model performance is evaluated using accuracy, precision, recall, and F1-score, supported by confusion matrix and precision recall curve analysis. The results show that both models achieve high accuracy of approximately 99%; however, accuracy alone is insufficient due to data imbalance. TabNet demonstrates higher precision (92%), indicating fewer false positives, while XGBoost achieves higher recall (72%) and F1-score (80%), reflecting a better balance between detecting fraudulent transactions and minimizing misclassification. These findings indicate that XGBoost provides more reliable overall performance for e-commerce fraud detection under imbalanced data conditions. The study contributes empirical insights into the comparative of ensemble learning and deep learning models for real-world fraud detection applications.