Stop A/B testing your hero. You don't have the traffic. Here's what to do instead.

The most common mistake I see on Activation Audits is a founder paying $99 a month for Optimizely to A/B test a homepage that gets 600 visits in a good month. The tool is doing nothing for them. The math literally does not work, and the alternative is faster, cheaper, and gives sharper answers.

This piece is the math. Then the alternative.

The math your A/B testing tool will not show you

A/B test sample sizes are not opinions. They are functions of three numbers: your baseline conversion rate, the minimum lift you want to detect, and your tolerance for getting fooled by noise. The industry default is 95% confidence + 80% power. Every credible calculator (Optimizely, Evan Miller, ABTasty, CXL) uses the same formula.

Peep Laja, who founded CXL Institute and has been the most-cited voice in conversion rate optimization for the last decade, summarizes the rule cleanly: 250 conversions per variation. Below that, your test is not a test. It is a coin flip with a longer name.

Translate that into traffic. At a 2% baseline conversion rate, hitting 250 conversions per variant means roughly 12,500 visitors per variant. With control plus variation, that is 25,000 total visitors per test cycle. At a 5% baseline you can cut that to about 5,000 per variant. At 1% you need 25,000 per variant just to detect a 50% relative lift, which is enormous. The exact numbers shift with the lift you are trying to detect, but the order of magnitude does not.

So when a marketing blog tells you "Sarah ran a test and got a 15% lift", the first question is: how many visitors did Sarah have. If the answer is under 25,000 and her baseline is 2%, she did not run a test. She got lucky and called it a finding.

Over half of SMB sites get fewer than 500 visits a month

BrightLocal ran the biggest public analysis of small business traffic in 2019: over 11,000 local business websites, Google Analytics data, real traffic. The finding (more than 50% of sites under 500 visits per month, average 414) was uncomfortable enough that most CRO tool vendors quietly stopped citing it.

That average of 414 is the key number. At 414 visits per month and a 2% baseline, hitting 250 conversions per variant takes roughly 60 months. Five years. Per test.

Most SMBs are running an A/B testing tool while sitting on traffic that mathematically cannot give them an answer in this decade. The tool is a comfort blanket, not an instrument.

Peep Laja called this out a decade ago

The reason CXL is the most-respected institute in CRO is that Peep Laja has been refusing to flatter founders since 2014. His "Don't do A/B testing if you aren't ready" rant is the single most-cited industry stance against premature testing.

The rule has three parts:

1. Get to 500 conversions per month minimum before you A/B test anything.
2. If you cannot, run customer interviews, heuristic audits, and 5-second tests instead.
3. Spend the tool budget on the alternatives. They give you better answers anyway.

I do not have anything to add to his rule. I am citing it because most SMB founders have never read CXL, and the people selling A/B testing tools have a strong financial incentive to not mention the math.

We hit a similar pattern in our recent piece on AI Overview citations: SMB-specific tactical advice almost always has SMB-specific math behind it, and the math gets buried because vendors selling the wrong tool to the wrong audience have a louder marketing budget.

Four things actually wrong with most SMB heroes

The reason your hero conversion is flat usually has nothing to do with the headline. It is one of four structural problems, in order of frequency.

1. Audience match

Your hero speaks to "businesses" or "founders" or "anyone who needs growth". Real conversion happens when one specific persona reads the page and feels seen. "Founders running B2B service businesses under $5M ARR" beats "founders" four times out of five. Specific names exclude the wrong people, which is the point.

2. Single offer

Most SMB heroes carry two to four CTAs above the fold: Book a Demo, Free Trial, Schedule a Call, See Pricing. Choice paralysis is real and measured. Cutting to one primary CTA reliably moves hero conversion 15 to 30% with no other change.

3. 5-second clarity

Jakob Nielsen and Microsoft Research both established that visitors decide whether they are in the right place within 10 seconds. CXL's clarity research tightens that to five. If a stranger cannot tell what you do, who it is for, and why it matters after a 5-second look, your hero is broken regardless of how clever the copy is.

4. Page speed

A 4MB hero animation that takes three seconds to load means your hero copy ships to a visitor who is already scrolling away. Speed is not aesthetic, it is structural. Hero needs to render in under 1.5 seconds on a mid-range phone. If it does not, none of the other fixes matter, because nobody is reading the copy you worked so hard on.

Fix any one of these and you get a bigger lift than six months of A/B testing the same hero would have produced.

The 5-second test: free, fast, devastating

Run this Saturday morning. Cost: $0. Time: 30 minutes.

1. Pick five people outside your industry. Friends, family, anyone who has never heard your pitch. Five is not arbitrary: Nielsen and Landauer's research shows 5 testers surface roughly 85% of usability problems; the next 10 give diminishing returns.
2. Show each of them your homepage hero on a phone for exactly five seconds. Use a kitchen timer.
3. Hide the screen. Ask three questions: What does this company do? Who is it for? Why should you care?
4. Score one point per correct answer per person, out of 15 possible.

If you score under 10, your hero is broken. The fix is not subtle. Rewrite it. Then run the test again until five strangers in a row get at least two out of three in their first five seconds.

Customer interviews beat split tests when n is small

CXL's own low-traffic CRO recommendation is to skip testing entirely and do customer interviews instead. The reason is simple. With 5 to 10 real customers describing how they think about their problem, you get the exact language your hero should use. Guessed copy almost never matches what customers actually say.

The protocol:

• Five to ten customers, 30 minutes each, recorded with permission.
• Three questions. How did you describe this problem before you knew solutions existed? What were you searching for when you found us? What made you trust us over the alternatives?
• Transcribe. Highlight verbatim phrases that appear three or more times across interviews.
• Use those phrases. Do not paraphrase them.

Voice-of-customer copy out-converts guessed copy by 25 to 50% routinely. That is a bigger lift than you can credibly extract from any A/B test you could run at SMB traffic levels, and it takes a week.

We use the founder brief template to structure these conversations. If you want, copy it. It is free.

Heuristic audit: what one trained eye is worth

A heuristic audit is one CRO-trained person walking through your site like a target user and scoring clarity, friction, distractions, and trust signals. It takes two to four hours.

The reason it works at SMB scale is that 80% of conversion problems are obvious to someone with pattern recognition: the form has 11 fields when 4 would do, the hero CTA is below the testimonials, the page has no trust signals, the mobile experience breaks on iPhones. These do not need a test. They need to be fixed.

The Activation Audit is our version of this. We diagnose all four Activation Stack layers in 72 hours, then hand you a fix list ranked by cost-to-impact. $99 to $149, and the fee credits to any build work that follows. If you want to see what one would cost to act on, our cost calculator walks you through layer-by-layer budgets in 60 seconds.

When you ARE allowed to A/B test

About 20% of SMB sites get above 1,500 visits per month per BrightLocal's distribution. If you are in that bucket and your baseline conversion is 3% or higher, you can credibly run about one test per quarter on a 6-week cycle.

What to test first:

1. Hero headline copy. Biggest possible swing. Test two genuinely different versions, not two phrasings of the same thing. "We rebuild SMB websites" vs "Your homepage is leaking 60% of your traffic" counts. "We rebuild SMB websites" vs "We redesign SMB websites" is a coin flip with a longer name.
2. Primary CTA copy. "Book a Demo" vs "See if it fits in 60 seconds" can move click-through 30%+. Real swing, fast result, easy to ship.
3. Single vs multi-step form. Multi-step almost always wins on lead-gen forms with five or more fields. Test it once, then never again.

What not to test at SMB scale:

• Button colors. The effect is 1 to 2% on a good day. You cannot see 1 to 2% with 1,500 visits per month.
• Font weights. Same problem, smaller effect.
• Hero image variants. Effect usually gets washed out by everything else moving in real traffic.
• Anything where you cannot articulate a hypothesis sharp enough to predict the direction before the test runs.

The default position should be: "I am not testing this. I am choosing it based on customer interviews and shipping." Testing is reserved for cases where you can articulate why this lift will be 20%+ and the test fits inside one quarter.

The hero rewrite framework we use on Activation Audits

When we audit an SMB hero, we rewrite it before the founder asks. The framework is three lines.

Line 1: Who it is for. "Founders running B2B service businesses under $5M ARR" beats "for everyone". Specific names exclude the wrong people, which is the point.

Line 2: What it does. The thing, in plain language. Skip the brand-y abstractions. "We rebuild your homepage so 5-second visitors get it" beats "We unlock growth through digital transformation".

Line 3: Proof or promise. A number, a guarantee, a timeline. "72-hour written diagnosis, $99 to $149" beats "Get in touch to discuss your needs".

That is it. 30 minutes per draft, write three drafts, pick the cleanest. Run the 5-second test against it. Ship. Do not A/B test it. You do not have the traffic.

If you want us to do this rewrite for you, the Activation Audit covers it. We diagnose, we rewrite, you ship.