Part IV - Terms - A/B Testing Flashcards
When is A/B testing a good idea?
When you have clear measurable goals and are testing specific theories
When is A/B testing a bad idea?
When goals are vague and designs have multiple variables
How many variations should be in A/B testing?
Two at a time to maintain clarity in results
What is a null hypothesis in A/B testing?
The assumption that any difference in outcomes is due to chance
How many visits are needed for significant A/B test results?
Enough to reach a statistical significance of 95% confidence
What is multivariate testing?
Testing multiple variables to see how they interact over time
Does A/B testing affect SEO?
No Google supports A/B testing as it does not count as duplicate content
How and when to interpret A/B testing results?
After reaching statistical significance to accurately reflect on the hypothesis tested
How many variables should be tested in A/B testing?
One at a time for clear actionable results
What should be tested in A/B testing?
Elements like CTAs headlines images and copy length
Can A/B tests be run on non-webpage elements?
Yes including emails PPC campaigns and CTAs
How to find A/B tests from similar companies?
Check websites like Which Test Won and Visual Website Optimizer for examples
What if A/B test results are not trusted?
Re-run the test to verify the results
How often should A/B testing be conducted?
Continuously as long as each test has a clear purpose
What is needed to start A/B testing on a site?
A/B testing software like HubSpot or Google Content Experiments
