Writing a Hypothesis You Can Lose

A hypothesis you can lose has three parts: the observation that prompted it, the change you will make, and the specific effect you expect, on a named measure, within a named period. For example: because recordings show visitors hunting for delivery costs, adding the delivery threshold beside the price will raise product page add-to-cart rate over the next four whole weeks. That sentence can be wrong, and its wrongness would teach you that delivery visibility was not the binding hesitation. A vague version, this will improve engagement, can never be wrong, so it can never teach. Write the losable sentence before the test starts, file it in the log, and judge the result against it, not against a story invented afterwards.

The first clause should cite evidence a sceptic could inspect: a step rate in the funnel, a pattern in recordings, repeated support questions, survey wording, or a usability session moment. It should not cite a trend, a competitor or a feeling. If the observation is thin, say so in the sentence, because a weakly grounded hypothesis deserves a cheaper test or a research step first. The guides on qualitative methods are the usual source of strong observations at this scale. Over time, the quality of your hypotheses becomes a record of how closely you watch your own visitors, which is the real skill testing is supposed to build.

The middle clause names the change narrowly enough that the result can be credited or blamed. Rebuild the product page is not a change; it is six changes wearing one coat, and if the rate moves you will not know which coat pocket held the money. Move delivery and returns beside the price is a change. Where a redesign genuinely bundles changes, be honest that the hypothesis covers the bundle as a whole and the lesson will be coarse. Coarse lessons are sometimes the right purchase at low traffic, since only large swings are readable anyway, a trade-off the pillar guide sets out. The rule is awareness, not purity.

The final clause commits to a measure and a window: add-to-cart rate on the product template, checkout completion, enquiry submissions that pass the fit check, over four whole weeks, compared with the baseline segment. Avoid promising a precise percentage you have no basis for; the commitment that matters is direction, measure and the smallest movement you would act on. Also name a guard measure, the thing that must not get worse while the primary measure improves, such as order value, refund rate or enquiry quality. A hypothesis without a guard invites wins that quietly tax another part of the business.

When the result contradicts the sentence, the sentence has done its job. Record the loss at full size in the test archive: the prediction, the numbers, and your best current explanation, labelled as an explanation rather than a new fact. A loss usually kills a theory, which is valuable, because that theory would otherwise have driven three more changes. What it must not do is get quietly rewritten into a near win. Teams that edit hypotheses after results soon have a perfect record and no knowledge. A one-person business has nobody to impress with the archive except its future self, who needs the truth.

Use this shape. Observation: what was seen, where, and how often. Change: the single alteration, with a screenshot or link. Expected effect: the measure, the direction, the smallest movement worth acting on, and the window in whole weeks. Guard: the measure that must not deteriorate. Decision rule: what happens on a win, on a loss, and on an inconclusive read, decided now. Five short blocks, written before launch, in the same place every time. The template matters less than the timing. A hypothesis written after the results are known is not a hypothesis; it is a press release, and your log deserves better.

Bad: the new homepage will perform better. Nothing is observable, measurable or losable in that sentence. Better: because exit surveys keep naming postage as the surprise, showing the full delivered price on the product page will lift checkout starts over the next four weeks, without reducing order value. Bad: the greener button will convert more. That is a cosmetic guess with no observation behind it, of the kind testing offers, not cosmetics warns against. Better: because trial users stall before inviting a colleague, adding a guided first project will lift activation among new signups this month. Each good example can lose cleanly, and that is the compliment.