Go beyond random AB Testing with the help of Psychology and User Testing

Contributed to this article
Gregorry Ciotti
Marketing strategist at Help Scout
Sabina Idler
community manager, technical writer & UXer @ Usabilla

AB Testing is awesome. It’s scientific, it’s easy to understand and explain and most of the times it has a very clear result. However it’s not the only way to optimize a website.

In fact, AB testing random things on your website will only get you random results. Adding some structure and research to your testing strategy will pay off big time. I have seen many optimization strategies that start with the AB test when in fact they should end with it.

Ever since I got my hands on Reading Virtual Minds by Joseph Carrabis, I understood that there is much more to conversion optimization than wild guesses and intuition. Contrary to the name, AB testing should not be about simple testing, it should be about confirming optimization theories you come up after analyzing existing data.

How can we take conversion and profit optimization beyond random AB testing? I decided to let experts with tons of experience answer these questions.

Research! Start with Psychology

I invite you to meet Gregory Ciotti, marketing strategist at Help Scout, and discover his blog, Sparring Mind, where he talks about behavioral psychology research on creative work, business, and life. Sounds complex?

Well, it’s supposed to be like that (it’s science after all) but Gregory makes it very easy to digest and understand.

Q: So Gregory, should marketers pay more attention to sciences like sociology or psychology when optimizing websites?

Gregory Ciotii, Marketing Strategist at Help Scout

Gregory Ciotii, Marketing Strategist at Help Scout

Absolutely. When it comes to selling, you’re always selling to people; I don’t care if you’re selling doggie treats, Fido isn’t going to be the one to lay down the bones for the purchase.

“The competent advertising man must understand psychology. The more he knows about it the better.”
Claude Hopkins

The reason that these behavioral sciences are applicable to things like CRO is because anything that helps persuade people online (or ‘increase conversions’) does so because it changes how people behave on your site. Sure, everyone is unique, but as I’ve argued on Sparring Mind, you better your chances when you tap into common cognitive patterns that researchers have tested time and time again.

As an example, I see many marketers argue over tiny details like button color and conversions, but they muddy the argument with guesses and anecdotal evidence, none of them understand the why. If they knew about the Von Restorff effect, they’d understand that when it comes to colors + conversions, what matters most is contrast since it helps break patterns and enhances recall.

If they knew about line of sight, they’d make sure their facial images and directional cues all pointed towards desirable outcomes (like an email form).

All in all, the behavioral sciences give marketers an incredibly insightful look into the “why” behind conversion bumps and drops; instead of relying on hunches, you can begin your testing by starting with proven behavioral patterns revealed by intensive research.

Q: Everybody nowadays has the skills/tools to implement an AB test but not everyone comes with a psychology degree. What skills/tools/resources do marketers need to start using psychology in their work?


Although I’ll slog through an interesting research paper, there is no better (or more accessible) way to learn about psychology than through great books.

And I’m not talking about Malcom Gladwell either… I’ve published a big list o’ books myself for people to get started. There are also a few great blogs that have raised the bar when it comes to publishing relevant academic research that relates to marketing; one of my favorites is the Neuromarketing blog by Roger Dooley.

All in all though, books offer amazing “rabbit hole” reads that can lead to other books, research papers, and a plethora of other sources for learning about psychology and human behavior.

What you can read in an hour often took a research months or even years to compile; there’s no better way to learn than that.

Analyze! Continue with User Testing

For small companies and startups, user testing is the meca of conversion optimization. Everybody would love to do it but lack of budget puts this method on nice to have.

I disagree that user testing can be done with very small budgets. You’ll say that inviting friends to navigate on your website or app while you ask questions and take notes it’s not scientific user testing, but I’ll say it’s better than not doing it.

Also, you can always turn to tools like Usabilla which makes remote user testing accessible and allows you to test your website with people you’ve never met before in their own environment. Sabina Idler is a community manager for Usabilla and i am glad to have her on the show. :)

Q: Sabina, how can marketers identify what’s wrong with their website, other than trying to guess A or B and test it?

Sabina Idler, Community manager, technical writer & UXer @ Usabilla

Sabina Idler,
Community manager, technical writer & UXer @ Usabilla

A/B testing is a great way to test small details on a page to see if that one detail can be changed to be more effective. However, this approach requires you to have a good idea about which parts of your website affect your conversion and also what changes might result in improvements.

Instead of endless trial and error, why not ask your visitors what they think of your site and what they would like to see improved?

For example, with tools like Usabilla Live, you can ask your website visitors for live website while they are browsing your site. Set up targeted surveys and find out why people don’t click your CTA, why they don’t sign up, or why they don’t follow through with their purchase.

The feedback that you get from real users will give you plenty of input for your A/B tests. Yes, you should still test if the gathered suggestions really increase your conversion, or if they are just individual preferences. But at least you have a good idea where to start.

Q: Isn’t user testing a very expensive method, available only to companies with big budgets?

User research can be as expensive as you want. Obviously, there are very comprehensive methods, such as in-person user testing, which cost both a lot of time and resources.

At the same time, there are plenty of remote research tools that are very accessible, even for small startups and freelancers with a small or no budget.

At Usabilla, we believe that remote user research is the future of user centered design. Why? Because it is quick, simple, and seamlessly fits into any timed development process. Besides, if you consider a few key concepts, anyone can do remote research.

Confirm! Do AB Testing

Yes, it’s time for AB testing. With well done homework, AB testing is not about guessing anymore: it’s about confirming your hypothesis.

From now on, betting on AB testing variants should be much more rewarding, literally. Time to place the bets.

Go beyond random AB Testing with the help of Psychology and User Testing by