All guides

What list crawling is in SEO

A finite URL set, fetched on purpose — not a spider wandering your site, and not Googlebot’s discovery loop.

The working definition

List crawling is an audit method. You give a crawler a known set of URLs — from a sitemap, a Search Console export, analytics, or logs — and it fetches only those addresses. It does not follow internal links. Depth stays at zero unless you raise it.

That is the opposite of a recursive spider crawl, which starts at a seed and walks every link it can find.

What Googlebot is not doing

Google Search has no product called “list crawl.” A sitemap is the closest official analogue: a suggested list, not a guarantee.

When the method earns its keep

Use a list crawl after a migration, to compare indexed URLs against a sitemap, or to re-check titles, canonicals, and noindex on a defined inventory.