---
title: "Handle keyword overlap and indexing on large sites | Ian Nuttall"
description: "Fix indexing issues with keyword cannibalization"
canonical: "https://ian.is/post/large-site-indexing"
language: "en"
---

# Handle keyword overlap and indexing on large sites

Jan 2024 / 51,459 views / [via X](https://x.com/iannuttall/status/1750933773242257610)

The most pages I've indexed on one site:

45,034,907

"but Ian, this site has absolutely tanked to just 1.34m pages", I hear you cry.

Yep - most people will never experience these indexing issues but I expect it at this scale.

It doesn't mean a site is dying, far from it.

This site is doing better than ever:

At this level, keyword cannibalisation is much more common and to protect resources, G might deindex your pages if they are ranking for very similar keywords.

The only real "fix" for this is to make sure you know which your most important pages are and focus on getting (and keeping) those pages indexed.

To do that:

\- Consolidate cannibalising pages
\- 404 or 410 pages you remove
\- Fix soft 404s
\- Block crawling of unimportant site sections

And finally - don't build sites with 3b+ fucking pages! 🤣

![Image from the original X post](https://ian.is/media/tweets/1750933773242257610/1750933773242257610/c8208610594ded95d96540e383c3dba45bb1e3fe2ad39ef61a776324a564d20a.jpg)

![Image from the original X post](https://ian.is/media/tweets/1750933773242257610/1750933773242257610/c487b69910c2bcdedeafdee5dcaf7477754e6b9fb6b6c044dc4d816f8ae49dc7.jpg)

second place was a very respectable 3,060,009 pages!

![Image from the original X post](https://ian.is/media/tweets/1750933773242257610/1750933773242257610/89bc0730af1699f52913140769da179e66bdbd4bd3bb1bba5c190499bb98f45c.jpg)

## Ian's List

Join 15k\+ subscribers