The Test Bench240 guides, 20 benches
Conversion.sale
The Test Bench - a field guide to conversion for small online businesses
Home / Bench 13 - Analytics and behaviour tools / Guide

Tracking Traps: Double Counts, Test Orders and Other Lies

The most damaging analytics failures are quiet: conversions counted twice, test orders inside revenue, consent losses mistaken for traffic falls. Learn the traps, verify on a rhythm, and distrust any sudden perfection.

Published 24 October 2026Evidence checked on 24 October 2026Small online businesses, global English edition

The short answer

Tracking rarely fails loudly; it fails by small, plausible lies. A purchase event firing on both the confirmation page and a tag, doubling conversions. Test and staff orders sitting inside revenue and rate. A consent banner change quietly removing a share of visitors from measurement, read as a traffic fall. Cross-device journeys counted as two strangers and one abandonment. Each trap produces numbers that look reasonable, which is what makes them dangerous: decisions get built on them for months. The defence is reconciliation on a rhythm. Your analytics outcome count should match real orders closely every month, and any sudden improvement in any metric should be suspected of being a counting change before it is celebrated as performance.

Double counts and the thank-you page

The classic trap: the platform integration and a manually added tag both send the purchase, or a customer revisiting the confirmation page re-fires it, and conversion rate quietly doubles on paper. Symptoms are a rate that jumps after a theme or app change and a conversion count persistently above the order count. Verify by placing one order and counting its appearances in the real-time and standard reports, the routine from analytics setup. Fix at the source, one sender only, and annotate the date range the error covers in your definitions document, because historical comparisons across that range are poisoned and future you needs the warning in writing, not in memory.

Test orders, staff orders and your own browsing

Small businesses pollute their own data more than any bot does. The owner tests the checkout weekly, staff buy at a discount, the developer clicks through every page after each change. On a large site this is rounding; on a store with sixty orders a month, six test orders are a tenth of the data. Filter internal traffic where the tools allow, tag test orders in the platform so they can be excluded from revenue reads, and keep a simple convention, a recognisable test name or address, so they can be spotted in any export. Above all, note testing days in the log, because a mystery spike explained in one line is a nuisance, while unexplained it becomes a theory.

Attribution traps: last click and the vanished middle

Source reports credit the last tracked touch, so brand search and direct harvest conversions that email, social and search created earlier, and channels get defunded for doing the top of the work. Cross-device journeys deepen the fiction: research on a phone, purchase on a laptop, recorded as a mobile abandonment and a desktop miracle. Do not solve this with heavier attribution tooling at small scale; solve it by reading source data as directional, checking it against the post-purchase discovery question in post-purchase surveys, and refusing to kill a channel on last-click evidence alone. The trap is not that attribution is imperfect; everyone sane knows it is. The trap is forgetting it at the exact moment a budget decision is made.

A verification rhythm that catches traps early

Monthly, reconcile three numbers: analytics conversions, platform orders, and actual paid orders net of tests and cancellations. They will never match perfectly, but their gaps should be small and stable; a gap that changes shape is the alarm. Quarterly, place controlled orders on phone and desktop and trace them. After any theme change, app install, tag edit or consent update, repeat the trace the same day, because that is when traps are born. Keep the reconciliation on the dashboard page itself, so the trust status of every other number is visible beside them. Data you have verified recently can carry decisions; data you have not is a rumour with a chart.

Distrust sudden good news on principle

The hardest trap is emotional: a metric leaps, and nobody wants to audit good news. Make it a rule that any sudden large improvement is checked against the reconciliation and the change log before it is reported, celebrated or built upon. Check whether a definition, a tag, a filter or a consent setting changed in the same week. This rule will occasionally delay a genuine celebration by a day. Far more often it will prevent the specific humiliation of presenting a doubling that was a duplicate tag, to a partner, a lender or your future self. The pillar for this bench, analytics and behaviour tools, puts trust before interpretation for exactly this reason.

Sources and evidence on the record

Evidence basis: Method guide. Any figure quoted is attributed to a named published source listed below; worked examples are labelled illustrations with round numbers, not measurements. Applies when: Every business, monthly, and after every site or tagging change

Next on the bench

Bench 13 - Analytics and behaviour tools Analytics and Behaviour Tools for CRO: The Working Guide Analytics answers where buyers are lost; behaviour tools show what losing them looked like. This guide sets up a small, honest measurement stack a one-person business will actually use every week. Bench 13 - Analytics and behaviour tools Analytics Setup That Answers Questions Instead of Collecting Numbers Set analytics up backwards: list the decisions you face, name the counts each decision needs, then configure only those events and reports. Everything else is storage you will never open. Bench 13 - Analytics and behaviour tools Consent, Privacy and Measurement That Still Works Honest measurement and lawful consent are the same project: ask properly, respect the answer, design reports that survive a share of visitors being unmeasured, and never let a banner break the visit it protects.