"Why is the traffic distribution not aligned with the test setting?" It's a common question from new users of ABConvert when they see the analytics number of their first test.
In this article, we'll delve into why this happens and how to address these discrepancies.
How ABConvert allocate traffic for tests
Our script do following things to allocate traffic.
Assign a random seed (range from 0 to 99) to customers when they first visit the store.
Store the seed in the customer's browser.
Allocate customers to specific groups based on their seed and the traffic distribution setting of the test.
Imagine you've set up a test, allocating 50% of your website traffic to the original group and 50% to the test group.
Here is a diagram to illustrate how it works behind the scenes.
The logic of allocating traffic is the same if you have multiple test groups in a test.
This approach not only provides randomness but also ensures revisiting customers will always stay in the same group during the testing period.
Why is traffic distribution not aligned with the test setting?
Filtering options
If you add filtering conditions in the test, traffic distribution will be influenced. We prioritize assigning visitors to their respective groups based on the set conditions.
For example, you set the traffic distribution to 50/50 and a filtering condition that assigns visitors from a specific traffic source (ex: utm_source contain facebook) to the test group.
In this scenario, we cannot expect traffic to be distributed as requested because an additional variable is added to the test.
There is no enough traffic in the test
ABConvert employs a random distribution mechanism to allocate traffic for all visitors.
However, it also means that in the short term, the traffic distribution may appear skewed or uneven.
Let's illustrate this with an example. Imagine you flip a fair coin 100 times. While the expected result is 50 heads and 50 tails, you might get 60 heads and 40 tails. As long as you continue to flip the coin, the distribution tends to even out, approaching the expected 50/50 split.
How much traffic is needed to have well-distributed traffic?
While there's no one-size-fits-all answer to this question, establishing a general guideline might help you navigate the process more effectively.
If you see the imbalance still exist after 250 views, please reach out to support for assistance.
Conclusion
In conclusion, it's crucial to recognize that fluctuations in traffic distribution are normal, especially in the initial stages of the test.
Instead of being alarmed by temporary imbalances, you should focus on monitoring the overall trend over a sufficient period. With time, the traffic distribution typically converges towards the specified percentages.