Handling Extreme Class Imbalance in Fraud Detection

Originally published at Riskernel. Fraud is one of the easiest machine learning problems to misunderstand because the target is so rare. In many portfolios, fraud is well below one percent of total events. That means a model can look excellent in offline evaluation while still creating a terrible operational outcome once it meets production traffic. If you are evaluating a fraud vendor or building your own stack, the first thing to understand is that this is not a standard classification problem. It is a rare-event decisioning problem with operational consequences. Why the base rate changes everything When fraud is extremely rare, “accuracy” becomes almost meaningless. Even AUC can look strong while the operating threshold behaves badly in the live queue. The real question is not “can the model separate classes in a notebook?” It is “can the model catch enough fraud at a threshold that does not drown the team in false positives?” Why good offline metrics can still mislead you A vendor