Why does an ROC curve matter?#
A ROC curve plots a model's performance on a single graph, making it easy to see the relationship between its accuracy in catching true positives (e.g., identifying a fraudulent transaction) and its rate of generating false positives (e.g., flagging a legitimate transaction as fraud). This helps you select the best-performing model and tune it to the level of risk your business is willing to accept.
How do you read an ROC curve?#
Imagine an AI model that predicts if a sales lead is "high-quality." The model assigns a confidence score (e.g., from 0% to 100%) to each lead. A business must then set a cutoff score to make a decision; for example, "treat any lead with a score above 70% as high-quality."
The ROC curve shows what happens at every possible cutoff score. The graph plots the percentage of high-quality leads that are correctly identified against the percentage of low-quality leads that are incorrectly labelled as high-quality. A curve that bows further towards the top-left corner of the graph represents a more accurate model, as it is better at correctly identifying leads without mislabelling the wrong ones.
How is an ROC curve different from an accuracy score?#
A ROC curve provides a comprehensive view of a model's performance across all possibilities. This is different from a single accuracy score (e.g., "95% accurate"), which can be misleading. For instance, a model that predicts equipment failure might be 99% accurate by simply guessing "no failure" every time if breakdowns are rare. The ROC curve reveals this weakness by showing exactly how well the model performs at its core task of separating the positive cases from the negative ones.