July 17, 2023

SEO Recipe | A guide on how to dedup your pages using Similar AI

Robin Allenson

Introduction

Are duplicate pages ruining your SEO? It’s time to find out.

There’s no need to spend hours trying to figure out the number of duplicate pages on your website. Even worse, you don’t want to stay in the dark about the pages that hurt your SEO.

In this blog, we will show you how to do deduping in just a few steps.

What do we mean by deduping

Deduping is the process of finding and organizing duplicate pages to ensure they are not cannibalizing your ranking opportunities. Deduping removes similar pages from your site. This increases the content uniqueness of your pages. In many cases, the content may be similar but one variable might still make the difference. It may even look like it’s similar when in fact, it’s different.

Either way, not having an idea of these can affect your SEO efforts.

SERP dedup extends deduplication by using search engine result pages (SERPs) to understand the meaning of similar page categories.

This is where you need to review the clusters and all the pages to understand all the existing and missing opportunities.

We wanted to simplify the process for you, so we’re ready to introduce our deduping process in just 6 steps.

SERP Dedup – Step by step

Here is everything you need to know to dedup in just a few steps.

  1. Pull in page ingredients 

First of all, you need to add the page ingredients in the Setup → Page Ingredients. This is where you add the pages you want to review. You can pull in pages from multiple sitemaps – and even multiple sites (especially useful when doing site migrations!).

  1. Extract keywords 

Now it’s time to choose the keywords you want to extract using the Workbench. The ‘extracted keywords’ are the keywords a page could rank for, whether it ranks or not. Often there are duplicate pages that don’t rank.

Click on Workbench → Pages and add a text operation to transform the H1, Title, URL, or other on-page text into the keywords you want to use.

Save the variable as $extracted_keyword.

  1. Run SERP crawls for extracted keywords 

This is the step where you’ll crawl the search engine results page (SERP) for the extracted keywords. Similar AI uses the SERP, along with our own AI, to represent the meaning of a keyword. If two keywords are similar and have very similar SERPs, they can be grouped together in the same topic. If two pages target the same topic, they can be deduped.

You’re still in Workbench → Pages. Click on Action → Get SERP

  1. Run Dedup

Right after the SERP crawl, it’s time to dedup. 

While still in Workbench → Pages, click on Action → Dedup.

  1. View duplicate pages

Switch from Workbench to the Opportunities tab.

In the Opportunities tab, you can view the duplicates and their summary figures. This is an easy way to share them with others in your team. 

  1. Save as an opportunity

In the Opportunities tab, you can review the duplicate pages. When you are ready, click on Publish.

What’s next 

Just like that, you are able to dedup your pages in 6 steps. 

Curious to try it on your own? Try Similar.ai today