A/B testing

A/B testing lets one published funnel serve different screen paths while keeping analytics tied to the same campaign URL. It is useful for testing the first screen headline, lead magnet, qualification order, or final call to action.

Configuration

Enable testing by setting ab_test_config.enabled = true and providing a variants array. Each variant needs a stable key and a numeric weight:

{
  "enabled": true,
  "variants": [
    { "key": "control", "weight": 50 },
    { "key": "short-form", "weight": 50 }
  ]
}

Weights are relative. A 70 / 30 split and a 7 / 3 split behave the same way.

* * *

Assignment

Variant assignment uses a deterministic hash of funnel_id and visitor_id, weighted by the configured split. The same visitor on the same funnel lands on the same variant across reloads and browser tabs, which keeps the experience stable and prevents one visitor from being counted in multiple variants.

Changing variant keys after traffic starts creates a new reporting bucket. Prefer keeping the original key and changing the screen content attached to it, unless you intentionally want to start a fresh experiment.

* * *

Screen targeting

Screens without an ab_variant_key are shared across every visitor. Screens with an ab_variant_key render only for visitors assigned to that matching variant.

A common setup is:

1. Keep the intro and outcome screens shared.
2. Create one or more variant-specific question or offer screens.
3. Keep the final form_collect screen shared so all variants convert through the same form fields.

Shared conversion screens make results easier to compare because each variant is measured against the same submission goal.

* * *

Reading results

Use the funnel analytics by_variant section to compare sessions, conversions, and conversion rate. Check the session count before declaring a winner, and inspect per_screen analytics to understand why a variant is winning or losing. A variant can improve early progression but hurt final submission quality if it attracts low-intent leads.

* * *

Guardrails

Avoid running multiple unrelated tests in the same funnel at once. If one variant changes the headline and another changes the form length, the analytics will not show which change caused the result. Keep experiments narrow, run them long enough to collect meaningful traffic, then archive or rename the losing screens once the winner is selected.