A/B Testing - which tool has helped you?

Question

What helps in A/B testing:

👊It needs to be run at the same time for both tests
👊Ensure the sample size is enough to tell if there was a statistically significant difference between these two variations
👊Ensure the website does get steady traffic in order for it to be statistically significant

Have you seen a tool that has helped you with A/B testing?

Fabian Maume · Accepted Answer

VWO is quite good to run A/B test. The free plan is enough for most A/B test. I also recommend to complement A/B test with qualitative analysis using a tool like clarity or Smartlook Qualitative Analytics

Marianna · Answer

@yukioyama Hello Yuki. It's essential to calculate the required sample size based on your experiment's specific requirements to ensure reliable results. Some experiments might require thousands or even millions of impressions to achieve statistical significance.

Yuki · Answer

Hey marianna these are great tips! Agreed that it's very important to keep things besides the variable of interest constant. What kind of volume are you typically looking for in a paid ads scenario (in terms of impressions) for statistical signficance?

Marianna · Answer

@fabian_maume Thankyou Fabian great tool! Looks similar to Posthog but in a friendlier UX.

A/B Testing - which tool has helped you?

Replies