What does an outcome check declare?#
Six things, fixed before work starts: the signal, how far it should move, who it applies to, how long to watch, what must not get worse, and how to separate this change from others shipping alongside it. The "must not get worse" list is what experimentation teams call guardrail metrics. Declaring any of it afterwards is how a disappointing result becomes a story about the wrong metric. At Microsoft, only about a third of tested ideas improved the metric they were designed to improve.
What happens when an outcome check fails?#
If a guardrail metric degrades, the system rolls back automatically, because rollback is a safety action and should not wait for a meeting. Deciding what comes next, whether to retry, change approach or drop the idea, stays with people.
The outcome check closes the delivery loop: hypothesis-driven development with the hypothesis written down first.