Split Testing Case Study: eCourse landing page optimization results

This is the third and final post in our split testing case study. If you want to review the last two posts click here and here.

Determining Our Split Testing Results

A split test is only as good as the interpretation of the results that come from it.

Analysing the results of your split test will help you determine if the change was a good thing or a bad thing and whether it should be implemented as the new baseline.

The thing about split tests is that you are mean to be running small experiments (in isolation) so that you can improve your desired results over time.

For Teach Yourself Websites the desired result was to increase signups on the home page’s subscription form for the eCourse.

If you recall in our last post, we discussed how we were tracking signups by looking at the signup location of the each user. This is a great metric because now we can compare, over the period of the test, how many people signed up on each page.

We’ve confirmed that similar amounts of people viewed each page with the original receiving 468 visits and the variation receiving 470 visits.

Ideally, we should let this test run for much longer to receive more visits, but this website doesn’t garner a lot of traffic. The traffic it does get comes from search engines and is quite focused though, so the sign up rates are quite high.

In other words, because the traffic is more relevant, ie. the are searching for courses on website creation, we can afford to judge our results on a smaller amount. This isn’t a rule of thumb though, make sure you are judging your landing page optimization by a statistically relevant number of visitors.

The Sign Up Results

The results from this split test were quite surprising to me. I was actually expecting that given more explanation about what the eCourse is about, the visitors would then choose to sign up more often. This was the assumption to be tested anyway, I’ll let the results speak for themselves:

Original Results

Split Test Results Original

Variation Results

Split Test Results Variation

As you can plainly see, the variation with the additional text resulted in less signups over the tested period. It can therefore be concluded that the original page is the winner with 12 more signups.

In fact, it performed 28% better than the changes we tested for implementation over the given time period.

Implementing the Winner

Normally with split testing you have to insert special javascript code to control the flow of visitors. When you’re ready to conclude your test like we are you have to go back into your testing console, or remove the code from your html templates. What a pain!

Simple Page Tester makes it really easy to implement the winner in a scenario like this. Now that I’ve determined which page is the winner I can click one button which removes all split testing on both pages, removes the losing page from the system, and cleans up the split test, making the winner the new master page.

Split Test Declaring The Winner

*click* and it’s all done for you with Simple Page Tester.

What Are You Split Testing?

Split testing is a scientific way to judge variations of pages for their effectiveness, it’s a tool that should be in every business person’s toolkit.

Simple Page Tester is the WordPress plugin of choice for split testing pages and posts.

We used it to run this case study and it made life a whole lot easier so we could focus on the important part: analysing the data and deciding if our changes were good or bad.

Without Simple Page Tester we couldn’t have known that this small change of adding a few extra paragraphs would have a 28% drop in subscriptions. It saved us from making this critical mistake.

So, now it’s time to ask yourself, do you know what impact the changes you are making has on your landing pages?