Skip to main content

A/B testing with audiences

Splitting an audience in two to test one change across two campaigns, and how the odd/even split actually works.

A/B testing in Bookboost is done with two audiences and two campaigns, not one campaign with two variants. The A/B testing filter is what splits your guests into the two halves.

Include gives you one half, Exclude gives you the other, and between them they cover everyone in your base audience exactly once.

Running a test

  1. Build your base audience as normal, for example everyone with marketing consent checking out this month.

  2. Duplicate it. See Duplicate an audience.

  3. On the first audience, add the A/B testing filter set to Include. This is group A.

  4. On the second, add the same filter set to Exclude. This is group B.

  5. Name them so you can tell them apart later, for example "Checkout month, A" and "Checkout month, B". Two audiences with near-identical names is how results get mixed up.

  6. Create one campaign against each audience, changing one thing between them.

  7. Compare the two campaigns' results.

Change one thing

Subject line, or send time, or the offer. Not two at once. If you change the subject line and the send time together and version B does better, you have learned nothing about which one caused it.

Keep the base audience identical between the two. Any other difference in filters and you are no longer comparing the message.

How the split actually works

The filter divides guests by whether their internal Bookboost profile ID is odd or even. Include takes the odd numbers, Exclude takes the even ones.

Those IDs are assigned in sequence as profiles are created across all Bookboost accounts, not just yours. In practice that lands close to half and half, but it will never be exactly 50/50, and on a small audience the gap can be noticeable. Check both audience sizes with Load preview before you send, and treat a result from a few hundred guests as a hint rather than a finding.

The split is also stable rather than random: the same guest falls on the same side every time, because their ID never changes.

What this does not do

  • It is not a random split. It is odd against even IDs, fixed per guest, so the same people are always in group A.

  • It does not pick a winner. There is no automatic result, no significance test, and no send-to-the-winner step. You read the two campaign reports and decide.

  • It is not built into a single campaign. Two audiences and two campaigns, every time.

  • The halves are approximate. Do not report a small difference between two unevenly sized groups as a result.

What to do next

Once you know which version won, use Audience filters: campaign behaviour to follow up with the guests who engaged, or to keep the losing group out of the next send.

Getting help

Open Help at the bottom of the left menu and choose Talk to Us, or email support@bookboost.io.

Did this answer your question?