Planning

To work out what you would need to see in order to believe something, before you go looking. Most evidence gathering starts with a method and works backwards, which is how teams end up with a lot of data and no decision.

It is worth an hour whenever a decision is expensive and the argument has become a matter of opinion. Naming the evidence in advance also names the evidence that would change your own mind, which is usually the more difficult half.

Start from the claim, not the research. Write what you are asserting, then what a reasonable sceptic would want to see, then what you would accept as enough.

Then choose the cheapest source that clears that bar. Often it is five conversations rather than a survey, or a number you already have and have not looked at. Write down what you will do for each possible result before you gather anything, because a plan written afterwards is a rationalisation with a timestamp.

The bar moves after the data arrives. You said forty per cent would be convincing, you got thirty-one, and the conversation becomes a discussion of sample quality. Sometimes that is a fair objection. It is only fair if you would have raised it at forty-nine.

The other failure is asking people to predict themselves. What somebody says they would pay, or would use weekly, is a poor guide to what they do, and a survey full of those answers is expensive confidence. Ask about what they did last time instead, and prefer a small piece of behaviour to a large pile of intentions.