XML sitemaps that actually help
XML sitemaps sit in a strange spot: endlessly recommended, rarely maintained, and almost never used for the thing they are genuinely good at, which is diagnostics.
This post walks through what a sitemap actually buys you, where the honesty matters, and the junk that keeps turning up in sitemap audits.
What a sitemap does and does not do
A sitemap helps Google discover URLs and gives it metadata hints, primarily lastmod. That is genuinely valuable on large sites, on sites with weak internal linking, and for fresh content you want picked up fast.
What it does not do: force indexing, override quality assessment, or boost rankings. Listing a URL in a sitemap is a suggestion that it exists and matters. Google still decides whether it is worth indexing. If your pages sit in 'Discovered, currently not indexed', the sitemap did its job; the content or the site did not.
lastmod, be honest or be ignored
lastmod is the one piece of sitemap metadata Google has confirmed it uses, with a condition attached: only when it proves consistently accurate. Google evaluates whether your dates are trustworthy, and once your CMS stamps every URL with today's date on every regeneration, your lastmod becomes noise and gets ignored sitewide.
Set lastmod to the last meaningful content change, not the last build, not a template tweak, not a footer year update. And skip changefreq and priority entirely; Google has said plainly that it ignores both.
Size limits and sitemap indexes
The limits are 50,000 URLs or 50MB uncompressed per file. Past that, you split into multiple sitemaps and list them in a sitemap index file. An index can only reference sitemaps, not other index files, so no nesting.
Reference the index from a Sitemap: line in robots.txt and submit it in Search Console. Gzip the files; the 50MB limit applies to the uncompressed size, but nobody needs to ship 50MB of XML over the wire.
Segment sitemaps for diagnostics
Here is the underused trick: even when you are nowhere near the size limits, split your sitemaps by page type. Products, categories, articles, locations, each in its own file.
Why bother? Because Search Console's Page indexing report can be filtered by sitemap. Suddenly 'the site is 71% indexed' becomes 'products are 96% indexed but locations are at 34%', and you know exactly which template to investigate. One sitemap gives you a number; segmented sitemaps give you a diagnosis. On large sites, this is usually the first restructure worth requesting.
Keep the junk out
A sitemap should contain exactly one kind of URL: canonical, indexable, returning 200. In practice, most sitemaps in the wild contain some mix of:
URLs that redirect, telling Google the sitemap cannot be trusted as a canonical signal.
404s and soft 404s, usually deleted content the generator never noticed.
Noindexed URLs, which is you asking Google to crawl things you then tell it to forget.
Non-canonical variants: parameter URLs, http versions, trailing-slash twins of the canonical.
Staging or dev URLs that leaked in from a shared config.
Every entry like this erodes the signal quality of the whole file. The check is quick: feed the sitemap URLs into Screaming Frog in list mode and crawl them. Anything that is not a 200, indexable, self-canonical URL comes out. Running this quarterly pays for itself; generators drift more often than you would think.
The ping endpoint is dead
If your CMS or deployment pipeline still pings google.com/ping with your sitemap URL, it is talking to a tombstone. Google deprecated the sitemap ping endpoint in 2023 and switched it off entirely. Discovery now runs through the robots.txt reference, Search Console submission, and regular refetching, with an honest lastmod doing the freshness signaling that pings used to fake.
For Bing and friends, IndexNow exists as a separate push mechanism. Google does not use it. Do not confuse the two.
A maintenance routine that takes an hour a quarter
Regenerate sitemaps automatically on content changes; a static sitemap is a lie with a timestamp.
Crawl the sitemap URLs in list mode and evict redirects, 404s, noindexed and non-canonical entries.
Diff the sitemap against a full site crawl: indexable pages missing from the sitemap are usually your orphan pages waving for help.
Check per-sitemap indexation ratios in Search Console and investigate the segment that dropped.
Verify lastmod dates against actual content changes on a sample of ten URLs.
None of this ranks a single page by itself. But a clean, segmented, honest sitemap makes discovery faster, makes indexing measurable, and turns Search Console from a dashboard into a debugger. For an hour a quarter, that is a deal.