Fraud Detection
An ML pipeline that scans more than 6.3 million transactions for fraud, in a dataset where guessing "not fraud" every single time already scores 99.87 percent.
The trap in the accuracy number
Fraud is 0.13 percent of this data. That one fact poisons the most obvious metric: a model that predicts "not fraud" for every transaction is 99.87 percent accurate and completely worthless, because it never catches the thing you built it to catch. Any honest version of this project has to start by throwing out accuracy as the goal.
94% recall and a 0.99 ROC-AUC — at a 0.13% fraud rate.Where a do-nothing model already scores 99.87%.
Getting to useful recall
I spent real time on exploratory analysis before touching a model, mostly to learn what fraud actually looks like in the data rather than assuming. The imbalance itself I handled with class weighting on a logistic regression, which tells the model to take the rare class seriously without blindly oversampling everything and distorting the distribution. Measured on all 6,362,620 rows, the model holds 0.938 mean recall across 5-fold stratified CV and 0.943 recall on the held-out test set, at a 0.990 ROC-AUC. Precision at the default threshold is only 2.2% — the cost of weighting for a class this rare — so the API returns the raw fraud probability rather than a hard block, keeping that tradeoff visible to the caller.
From local demo to a live endpoint
What started as coursework with a local model and a Streamlit demo is now a cloud-deployed HTTPS scoring API: training and model registry on Azure ML, served from Azure Container Apps. The local run and the Azure ML job produce identical numbers — deterministic, logged via MLflow — so what you see below is exactly what's running.
What was actually hard
It was never the model. Fitting a classifier is the easy part. The whole difficulty lived in the class imbalance, in getting a system to reach useful recall when the signal it needs is a tenth of a percent of everything it sees. That is where the deliberate work went, and it is the part that transfers to the next imbalanced problem.