At Google and Bing, only about 10% to 20% of experiments produced positive results. Here's how a smaller business can run ad tests that teach something either way.
A new ad runs for three weeks and does worse than the old one. The usual reaction is that the ad failed, the agency failed, or paid search doesn't work for this business. Usually none of that is true. One idea lost, and now you know something you didn't know before.
A line that circulated on LinkedIn drew the distinction well: "Losing doesn't make you a loser. Quitting does." The people with the most data on this would agree.
In a 2017 Harvard Business Review article, Ron Kohavi and Stefan Thomke reported what happens when large companies test their ideas properly. Most of the ideas don't work, and experts often misjudge which ones will pay off.
One caution belongs beside those figures. They describe very large companies with huge traffic, testing product and website changes, and the numbers are from 2017. They don't predict your win rate. The lesson is the direction: if the best-resourced teams lose most of their tests, a losing ad is ordinary.
A small business won't reach the certainty those companies get, because it doesn't have the traffic. It can still avoid the common trap, which is changing several things at once and then not knowing why anything happened.
| Part of the test | Teaches you something | Teaches you nothing |
|---|---|---|
| The change | One thing, such as the headline | New headline, offer and page together |
| The win rule | Written before launch | Decided after seeing results |
| The measure | Calls, forms or sales | Clicks alone |
| The record | A dated note of what happened | Someone's memory |
Stopping one ad is normal. Stopping the testing is the real loss, because the campaign then has nothing left to improve it. A sensible rule is to retire the idea, keep the campaign, and start the next test from what the loser showed you.
Sometimes a campaign should end. If several well-run tests over a fair period all fail, and the numbers show the offer can't pay for itself, walking away is a decision based on evidence. That's different from quitting after one bad month.
How WebKing runs this
We run ad changes as planned tests with the win rule written first, so a loser still tells us what to try next.
There's no single figure for ads. Harvard Business Review reported that only about 10% to 20% of experiments at Google and Bing generated positive results in 2017, which shows that most ideas lose even at very large companies.
Compare it against the rule you wrote before launch, using calls, forms or sales rather than clicks alone. If you didn't set a rule first, you'll struggle to judge the result fairly.
Usually not. Retire the idea and keep the campaign. Consider ending a campaign only after several well-run tests over a fair period show the offer can't pay for itself.
Change one thing. If you change the headline, offer and landing page together, you won't know which one made the difference.
Sources
The Lab is original analysis by WebKing. We summarize and interpret developments from the sources above for industrial, commercial, and small business owners. Figures are reported as published by their sources.
More from the desk
Photo: Ze VieiraMicrosoft is testing inline color selection in Bing Shopping results. If it rolls out, your product images and availability just got more visible before customers ever land on your site.
Photo: Rubaitul AzadA fresh validation report lets you catch missing costs, clicks, and impressions from your non-Google ad platforms before they pollute your performance metrics.
Photo: MailchimpOn August 17, Google is reining in Performance Max campaigns that have beaten their targets. Here's what to adjust now to keep your efficiency gains.
Free audit
We read your search, ads and AI visibility, then tell you which of these moves actually applies to your business. Free, 30 minutes, no pitch deck.
