Monday, January 27, 2014

The A/B Testing Process

The correct way to run an AB testing experiment (or any other experiment for that matter) is to follow the Scientific Method. The steps of the Scientific Method are:

  1. Ask a question: "Why is the bounce rate of my website higher than industry standard?"
  2. Do background research: Understand your visitors' behavior using Google Analytics and any other analytics tools running on your website.
  3. Construct a hypothesis: "Adding more links in the footer will reduce the bounce rate".
  4. Calculate the number of visitors/days you need to run the test for: Always calculate the number of visitors required for a test before starting the test. r 
  5. Test your hypothesis: You create a site wide A/B test in which the variation (version B) has a footer with more links. You test it against the original and measure bounce rate.
  6. Analyze data and draw conclusions: If the footer with more links reduces bounce rate, then you can conclude that increased number of links in the footer is one of the factors that reduces bounce. If there is no difference in bounce, then go back to step 3 and construct a new hypothesis.
  7. Report results to all concerned: Let others in Marketing, IT and UI/UX know of the test results and insights generated.

No comments:

Post a Comment