Customer Segmentation Engine

Problem & context

Built on 14 months of real Caterpillar-dealer aftermarket sales data for (Africa, dealer K600) the same warehouse behind the TAE Analyst Agent project. The sales team had 2,635 customers and no way to prioritize them beyond gut feel: who to protect, who to grow, who's worth a phone call.

Classic RFM assumes two things that don't hold here. There's no daily granularity every invoice date is a month-end reporting rollup, so recency has to be measured in months, not days. And the source export was capped at exactly 1,000,000 rows for three straight years, which didn't just clip line items, it dropped whole customers. Checked against the one clean stretch of data available (the untruncated 2025 rows), that cap cost the 2024 slice 84% of its true revenue and about two-thirds of the customers who'd actually bought something.

Technical approach

The fix is a per-customer data-sufficiency tier (verified against the raw warehouse: 230 dense / 1,398 clean / 143 partial / 864 unverifiable), carried through every table, chart, and export so a floor revenue figure from a fully-observed 2-month window never gets compared directly against one from a truncated 12-month window.

The clustering side is a full sweep, not a single algorithm assumed correct: 792 candidates across 7 families (centroid, hierarchical, distribution, density, graph, mixed-type, and an autoencoder), 2 feature sets, 3 skew transforms, k = 2 to 10, all logged to MLflow. Two disqualifications mattered more than the winner:

• An autoencoder that scored 0.685 silhouette in its own embedding space and 0.145 in the original RFM space — the overfitting check most segmentation write-ups skip.
• A Ward hierarchical model whose silhouette looked competitive but collapsed under bootstrap resampling (0.565 mean ARI against a 0.75 floor).

KMeans, k=4 on log1p-transformed RFM, was the only finalist to clear that floor (0.970 mean ARI, worst of 100 resamples still 0.800).

Results

Four segments: Heavy Hitters (7% of customers, 86% of revenue), Steady Regulars (29%, 12%), Long Quiet (21%, 1%, and 95% built on unverifiable data reported honestly, not treated as a real customer type), and One-and-Done Buyers (44%, under 1%).

The reactivation list splits the same way the data-quality system demands: 45 confirmed-dormant customers worth $668,810 versus 216 apparently-dormant customers worth $4.9M whose silence can't be told apart from the export bug 7.4x the size, and never shipped as one combined number.