Applied ML ยท August 2026

Worth Mailing

A charity sends 82,000 mailers at $2 each to households that might donate, ranked by a model's predicted probability. That only works if a score of 0.8 really means an 80% chance of donating. This project measures that gap on the KDD98 charity mailing dataset, corrects it with three calibration methods across three model families, and checks whether the fix is worth the compute it costs to produce.

Same dataset, three models, three fixes

Logistic regression, random forest, and XGBoost are trained on the same 82,318-household KDD Cup 1998 dataset, then each is corrected with Platt scaling, isotonic regression, and temperature scaling, a 3x4 grid of before-and-after Expected Calibration Error. Logistic regression comes out well-calibrated on its own; the tree-based models don't, and not by the same amount or in the same shape.

Three histograms comparing donors and non-donors across prior gift amount features, the strongest predictors of donation
The three features most correlated with donation, split by class, all prior gift amounts. Most of the separating power lives in gift history, not demographics.
Isotonic regression cut the random forest's ECE from 0.0334 to 0.0046.Platt scaling helped the same model, but made XGBoost's calibration worse.

Pricing the fix, not just scoring it

A lower ECE number doesn't mean anything to whoever approves the mailing budget, so every calibration result gets translated into a simulated mailing run. At 10,000 pieces, the isotonic-calibrated contact list cost $84.61 less than the uncalibrated one for the same donation revenue; at 100,000 pieces that gap scales to $846.10. Every training and calibration run is also logged with CodeCarbon, so the calibration gain is weighed against the energy it cost to produce, not just reported on its own.

scikit-learnXGBoostCodeCarbon