CASE STUDY - B2B MOBILITY PLATFORM
A churn model that held up outside the lab.
A global business-travel platform flags accounts at risk of canceling three months out. NEXUS matched a heavily-tuned model on accuracy - and held that accuracy on new data far better.
The Task
Flag accounts likely to churn, 3 months out
Accuracy Gain
10% more churn caught
Generalization
Far less overfitting
A well-tuned model that looked better on paper than in production
The platform's team had built a gradient-boosted model to catch accounts about to churn - usage patterns, billing history, support tickets and admin activity all folded into a carefully tuned classifier. It scored well on the data it was trained on, but that score didn't fully carry over once the model saw fresh accounts it hadn't trained on: a classic sign of overfitting, and a risk in any model asked to catch problems before they happen.
Same account data, no manual tuning
NEXUS's pre-trained foundation model was given the same account-level data the existing model used - usage, billing, support, and account-management signals - with none of the hand-tuning that went into the original build. Both were then scored the same way: trained on one stretch of history and checked against a later, unseen stretch of accounts and outcomes.
More accurate on new accounts, not just familiar ones
Measured Outcomes
+10%
better at catching real churn cases without false alarms (PR-AUC: 0.509 vs. 0.463) on accounts it hadn't seen
5%
drop from train to test accuracy for NEXUS, vs. a 29% drop for the existing model
0.914/0.901
accuracy at telling at-risk accounts apart from safe ones (AUC) on held-out accounts - NEXUS vs. the existing model
On the accounts that matter most - the ones neither model had seen before - NEXUS came out ahead on every measure checked: how well it separates at-risk from safe accounts, how well it catches real churn without crying wolf, and how consistent its scoring is. The existing model's edge on familiar, previously-seen data didn't carry over nearly as well to new accounts, a 29% relative drop in accuracy from training data to the real test. NEXUS held its accuracy far more consistently, ending up 10% better where it counts: catching churn on accounts it's never seen.
A model that trusts new data, not just familiar patterns.
The team's original build was thorough - dozens of engineered features, careful class-imbalance handling, an extensive tuning pass. NEXUS reached a more reliable result without that setup, on the same raw account signals. That matters for a churn model specifically: it has to be right about accounts it has never seen, months before they show obvious warning signs, and a model that just memorizes the past is the one most likely to miss them.
