What Is a ROC Curve?
A ROC curve is a visual tool used to evaluate the performance of an AI model that makes a "yes or no" decision. Businesses need to understand how accurate a predictive AI model is, especially when there's a trade-off between catching important events and creating 'false alarms'. A simple accuracy percentage can be misleading and doesn't capture this critical balance.
How it helps#
A ROC curve plots a model's performance on a single graph, making it easy to see the relationship between its accuracy in catching true positives (e.g., identifying a fraudulent transaction) and its rate of generating false positives (e.g., flagging a legitimate transaction as fraud). This helps you select the best-performing model and tune it to the level of risk your business is willing to accept.
How it works#
Imagine an AI model that predicts if a sales lead is "high-quality." The model assigns a confidence score (e.g., from 0% to 100%) to each lead. A business must then set a cutoff score to make a decision; for example, "treat any lead with a score above 70% as high-quality."
The ROC curve shows what happens at every possible cutoff score. The graph plots the percentage of high-quality leads that are correctly identified against the percentage of low-quality leads that are incorrectly labeled as high-quality. A curve that bows further towards the top-left corner of the graph represents a more accurate model, as it is better at correctly identifying leads without mislabeling the wrong ones.
How it is different#
A ROC curve provides a comprehensive view of a model's performance across all possibilities. This is different from a single accuracy score (e.g., "95% accurate"), which can be misleading. For instance, a model that predicts equipment failure might be 99% accurate by simply guessing "no failure" every time if breakdowns are rare. The ROC curve reveals this weakness by showing exactly how well the model performs at its core task of separating the positive cases from the negative ones.