While the science and tools of split testing are not perfect by any means, my 20+ plus years of experience in UX/design was challenged on many occasions to explain real-time user behavior.

Until the last decade or so, we didn’t fully qualify the User Experience. We didn’t truly quantify what worked and what didn’t for the user. Many in the industry made assumptions based on common accepted design & development architecture.

The average industry win rate across split testing is typically about 20%. That means that only 1 in 5 tests may produce usable results. We tend to measure program success on only the winning tests but what happens to all that other data?

There’s a lot of talk about the effectiveness of social proof psychology in split testing. Interestingly, the more it’s talked about the more it may be true. That’s because social evidence we hear from others in our culture persuades us to assign more credibility towards that information, whether or not it’s actually true.