What is a Multivariate Test?

A multivariate test is an experiment in which several elements on a page are tested at the same time in different combinations. Unlike in an A/B test, you test several variables at once to find out which combination works best.

The difference between an A/B test and a multivariate test

In an A/B test you compare one element in two variants. In a multivariate test you test several elements (for example the headline, image and CTA text) in all possible combinations. A multivariate test needs considerably more traffic for statistically reliable results: at least 10,000 visitors per month on the test page.

When do you use a multivariate test?

Only when you have enough traffic for reliable data. For lower volumes an A/B test is more effective and reaches reliable results faster. Multivariate tests are useful when you want to understand how elements interact with each other.

Multivariate testing and CRO

Multivariate testing is an advanced CRO technique. Use tools such as VWO or Optimizely. Use heatmaps and session recordings beforehand to identify the most promising elements.

Is a multivariate test harder than an A/B test?

Yes. The analysis is more complex and needs more data and time for reliable conclusions.

Can I start with a multivariate test without A/B testing experience?

Always start with A/B tests and build up experience from there. More at developers.google.com/search.