Why A Losing Test Isn’t A Loser And Could Be Costing You 35% Increases

It is important when split testing that you don’t get discouraged by a losing test.  A losing test is simply an opportunity to learn something and build upon that to make the next one a winner.  You must embrace the process of optimization and in the end, you will usually win the race. If you simply test random elements and keep bouncing around you won’t learn anything and will end up getting frustrated.

We have a perfect example of this in this special case study.  
Here we have a skincare company and their best selling products which happens to be a series of 3 different kits for different skin types.  The products are amazing and created by a world-renowned dermatologist.

They were, however, having a bit of an issue in scaling the sales of this product and getting over a plateau of growth.

The original page, listed below highlighted the product steps and a little more about the process and how it works.  Lower on the page they had each of the 3 types of products including a buy button next to each and that was basically the end of the page. There was no proof and it was actually confusing as to which product they should choose and what truly the next step was.

Note that this product is a luxury product with a pretty healthy price tag so we felt that it was important to further sell the product and build the value to justify the price tag.  On the original page, they didn’t reveal the price so the shock was actually a pretty big one if they did click through to the checkout.


We spent a considerable amount of time gathering heatmap data and going through all the analytics of the page to come up with ideas to make it better.  The outcome was that it would be more beneficial to just try a re-design of the page versus trying to get incremental improvements.

The first re-design looked like this…


What we did was added better imagery, a clear benefit-driven headline and social proof in two different areas.  For the “how it works” section, we supported them with video and really put the emphasis on the benefits of the product and science behind it.

Instead of having all 3 different products in this line highlighted separately we added a large image of the product and leveraged a dropdown. You will notice we also revealed the price on the page.  Plus to help them choose we added a link with a popup that explained more about the differences in the 3 options.

Much to our surprise though this new design didn’t beat the current control. In fact, it reduced conversions by quite a large margin.


Discouraging, to say the least, given the amount of time it took to create this particular test.  But after looking at the heatmaps and combing through the data again we noticed a few hang-ups.

Instead of scrapping the idea and moving onto something else we decided to iterate and test it again.
We replaced the headline with the name of the product with supporting information and benefits.
Instead of the drop-down menu we leveraged radio buttons.


And changed the “add to bag” button label to “Buy Now” to emphasize the next step is to make a buying decision.

The results this time were much different than the first test.


35.4% increase in sales and over 41% increase in revenue per visitor.

Now we still have a little bit of time left on this particular test but it has stayed steady with the improvement rate and we feel confident this will be our new control to beat.

Now I ask you… are you paying close attention to your data and making changes to losing tests to get winners or are you just giving up?