For SEO strategists, it is sometimes difficult to know which of the many changes we make to our websites actually impact the overall SEO performance. For example, we may don’t know whether a change to a page’s title has a positive or negative effect on rankings and clicks. 

One way to bring that quantitative rigor to your organic search efforts is to do SEO A/B testing.

In this guide, we’ll explain what is SEO A/B testing, how it is different from CRO A/B testing, the SEO elements you can split test, and how to perform an SEO A/B test successfully.

Let’s dive in.

What is SEO A/B Testing?

SEO A/B testing is the optimization of web page elements through before and after testing. The goal of SEO split testing is to get more people to click on your link in Google’s search results and visit your site.

The Differences Between A/B Testing for SEO Versus CRO

A/B testing for SEO is a bit different than what we would do for conversion rate optimization or user experience testing. In this part, we’ll aim to shed light on each technique, how they differ, and what they’re used for.

Aims and Objectives

Although CRO and SEO testing both use A/B testing, their goals – and how they are carried out – are very different. CRO aims to improve the conversion rate once visitors have reached the site; SEO focused testing aims to increase the number of visitors to the site, by implementing technical changes that will improve the website’s position in the SERPs.

Controls and Variants

In SEO A/B testing, as we have seen, pages are split into two sets—controls and variants. Changes are made only to the variant pages, and data collected are compared to a previously determined performance forecast.

In CRO A/B testing, two versions of the same page are set up and run live at the same time, and human users are redirected to a random one. Thus, it is the users themselves that are split into controls and variants. Data is collected from how users interact with the version of the page they are assigned to and compared.

Versions of The Pages

In SEO split testing, there is only one version of each page at any time. With Google as the determiner of how these pages will rank, duplicated content (which Google is against) is not encouraged.

CRO A/B testing, on the other hand, requires two versions of the same page to be live at the same time so data can be collected and compared.

In summary, there are several key differences between CRO and SEO A/B testing. Both tests are important steps in improving online performance but target very different metrics.

What Types of Websites Can Run SEO A/B Testing?

Even though technically anybody can participate in SEO A/B testing, there is a set criterion that websites are recommended to have met in order to reap the best results.

Firstly, websites being tested should have sufficient and stable organic traffic in the first place. This usually points to websites that have a few tens of thousands of organic sessions, so that when testing is in place, changes can be easily spotted and can easily acquire statistical significance.

Secondly, websites being tested should have at least hundreds of pages in a similar template for testing. This is because half of them will have to be set aside as ‘variant’ pages during testing and the other half will have to be kept as ‘control’ pages. So the more pages you have available, the easier it will be to collect enough data and rule out anomalies.

Although not an exhaustive list, some example types of sites that are good for testing are:

• Travel
• E-commerce
• Sites with lots of local pages
• Retail
• Recruitment
• Real estate
• Publishers
• Listings websites (eBay, Craigslist, events)

What Can I A/B Test?

Anything can be A/B tested on our websites, but for SEO purposes, certain site elements are more likely to result in keyword rankings improvements because of the weight they carry in Google’s algorithm. For that reason, the below elements are the best use cases for SEO A/B tests.

1. Title tags

Choosing title tags is so important and has a huge impact on search results. Title tag changes are very impactful from a rankings perspective because they directly influence click-through-rate (CTR). Google has a normalized expected CTR for searches, and if your landing pages continually fall below the mark, it will negatively impact your overall chances of ranking.

2. Meta descriptions

For those websites that already have a lot of keywords on page one and are therefore getting lots of impressions, A/B testing meta descriptions can be really beneficial. Like title tags, they directly impact CTR, and improving them can result in significantly more clicks and thus better rankings. 

3. Schema markup

If you can, it’s good to add schema markup to all of your web pages, but if certain pages on your site still don’t have schema.org markup, adding it can be a great use case for an A/B test.

4. Internal links

Internal linking tests require careful planning because as well as impacting the pages that you change, they affect the target pages that gain or lose internal links. You need to measure the impact on multiple groups of pages at the same time. Some examples of internal linking tests are

• Internal anchor text tests
• Adding or removing links to related products
• Increasing or decreasing the number of links to related products on shopping websites
• Linking to deeper pages in a site’s architecture

5. New Content

Adding quality long-format content to your landing pages is always beneficial for SEO because longer content implies topical depth. With a well-thought-out SEO Copywriting strategy we can create truly comprehensive, helpful content for your users, which we can then test to measure whether Google positively responds to those quality signals.

How to Perform an SEO A/B Test? 

Given Google will only ever show one version of a page in the search results, the majority of SEO tests are technically ‘before and after’ tests rather than true A/B testing. The exception is where you are making changes to broadly similar sets of content, keeping a control group and comparing the change in performance of your test candidates against the control.

Here’s how to run a meaningful SEO A/B test.

1. Select Testing Pages

First, you must select the group of pages on your website you want to test on.

These must be pages in the same template. These pages could be blog posts, product pages, category pages, or any other types of pages that are of the same type. Moreover, it is optimal that there are at least hundreds of them as it will make data collection and comparison much easier.

The closer these pages are to each other in terms of format, and the more pages there are for testing, the better.

The pages should also have historical data of a few hundred days, preferably at least a year, so that any trends in traffic throughout the year have been stabilized and can be used in data forecast and analysis.

2. Create A Hypothesis

Next, you must create a hypothesis for your A/B test.

A hypothesis is a prediction of the results of your experiment. Since your goal is to increase visibility on SERPs, the best SEO A/B hypotheses are grounded in what you already know about the search engine you want to rank on, and what you already know about your own web pages.

3. Separate Pages into Controls and Variants

With a solid hypothesis founded, you can start dividing your test pages into controls and variants.

How you divide them is up to you. However, it is important to refer to historical data of how the pages have been performing and make sure each set of pages have similar trends and levels of traffic. It would also certainly make sense to consider external influences that could impact the results of your data and plan accordingly.

4. Implement Changes

Next, you can implement the changes you had in mind onto the ‘variant’ pages and leave the ‘control’ pages unchanged.

This leaves you with two sets of live pages that adhere to two different formats at the same time.

5. Gather and Analyze Results

After implementing changes, trends in data can start emerging within a week if there are no complications. However, it is expected that SEO testing should take 2-4 weeks for these trends to gain statistical significance and stabilize.

After sufficient time has elapsed, measurements such as the actual number of clicks from SERPs, and click through rate of both the control and variant pages can be taken and compared to those of their respective forecasts.

The Bottom Line

Split tests for SEO can fill in the gaps in your SEO strategy and ensure that your content resonates with potential customers.

Now that you’re ready to take on your first SEO A/B split test, have you decided what variables you’ll experiment with?