The Leaking Sitemap: How Test Pages End Up in Google – and in AI Search

There is a search we run in the first ten minutes of every audit, and it is not sophisticated. Type site:yourdomain.com into Google and read the results – all of them, not just the first page. It costs nothing, requires no tools, and as first-pass checks go it is one of the quickest we know – and a dependable way to make a marketing director go quiet. It is not a complete inventory, though: treat the results as a sample, and confirm what you find in Search Console, your sitemap reports and crawl data.

Because what turns up, more often than anyone would like, is not the polished site the firm believes it publishes. It is a page titled “Test – please ignore”. A staging version of the homepage with lorem ipsum in the banner. Three near-identical drafts of a service page, all live. A campaign landing page for an event that happened two years ago, still promising early-bird pricing. An internal calculator the development team built for a client demo. All indexed, all public, all quietly attached to your firm’s name. This is one entry in a longer catalogue of ways professional-services websites undermine themselves in AI search, but it deserves its own article, because it is the one we find most often and the one that takes least effort to fix.

The finding: your sitemap is an invitation list

At a firm we audited recently – a professional-services business with a well-maintained site, or so everyone believed – the site: query surfaced pages nobody in the marketing team knew existed. The developers knew, vaguely; the pages had been “temporary”. That was some time ago.

The uncomfortable part was not that the pages were indexed. It was that they were listed in the sitemap. A sitemap.xml is not a passive inventory – it is the file you hand to search engines and any other crawler that discovers or processes it, saying: here is everything worth reading about us. In practical terms, a sitemap is a public discovery list – not a permission system. The chain from there is short: the sitemap invites crawling, crawling leads to indexation, and whatever ends up indexed becomes part of the information available for retrieval – in classic search results, and by the AI-powered search systems that build their answers on top of the indexed web. A test page in your sitemap is not lurking where nobody will find it. It has been formally introduced.

Two clarifications keep the metaphor honest. A URL’s presence in the sitemap is not a guarantee it will be crawled or indexed – engines treat the file as a hint, not an instruction. And removing a URL from the sitemap does not remove it from Google: if the page is discoverable anywhere else – a link, an old reference, its own history in the index – it can stay indexed regardless.

Run the query on your own domain before reading further. We will wait. If everything that appears is a page you would happily show a prospective client, you are in a small and fortunate minority.

How the leak happens

Nobody decides to publish “Test – please ignore” to the world. The leak is a system failure, not a personal one, and it happens the same few ways almost everywhere:

• The sitemap is auto-generated from “everything published”. Most CMS and SEO plugins build the sitemap from every live page by default. Publish a page – for any reason, for any audience – and it is in the sitemap within the hour. The sitemap is doing exactly what it was configured to do; nobody re-examined the configuration.
• Staging content gets published to production “temporarily”. A developer needs to test something under real conditions, so a page goes live “just for an hour”. The task that follows the test – taking it down – has no ticket, no owner and no deadline, so it survives. Temporary is the most permanent state on the web.
• robots.txt is assumed to hide things it does not. The most common misunderstanding we encounter, including among developers. Blocking a URL in robots.txt stops crawling, not indexing. If Google already knows the URL – from the sitemap, from a stray link – it can keep the page in its index, sometimes with the memorably unhelpful note that no description is available. The door is locked, but the page is still on the guest list.
• noindex is applied after indexing, and nobody waits for the recrawl. A noindex tag on an already-indexed page does nothing until Google revisits and reads it – which, for a low-priority test page, can take weeks. Teams apply the tag, consider the matter closed and never verify.
• Nobody owns the checklist. Marketing assumes the developers handle indexation; the developers assume the SEO plugin handles it; the plugin is faithfully executing a default nobody chose. Every leaked page we have ever found traces back to this: not a mistake, but a gap between responsibilities.

Why it matters more in 2026

Ten years ago, the case for cleaning this up was tidiness and crawl efficiency – real, but easy to deprioritise. That calculus has changed.

When an AI engine assembles an answer about your firm, it builds context from your own pages. It does not distinguish “real” pages from “leaked” ones – it has no way to know the duplicate draft was an accident or the broken test page was never meant for anyone. Everything indexed under your domain can be interpreted as publicly available material associated with your firm, whether or not your team intended to publish it. A half-built page with placeholder text in that context is a trust signal working against you, in a medium where you never see the assessment being made.

And the weight your own site carries in that assessment is not speculative. BrightLocal’s analysis of AI search sources found that “the vast majority of sources across every single LLM and industry were businesses’ own websites” (BrightLocal). Read that finding alongside the leak described above and the conclusion is blunt: your index hygiene is your AI presentation. The firm that lets test pages sit in its sitemap – and, in due course, in the index – is briefing the systems that describe it with its scrap paper.

One further place this bites: leaked pages survive domain moves. When a firm rebrands or merges, the redirect map gets built from the pages everyone remembers – and the forgotten test pages ride along, or break loudly on the new domain. It is one more reason the rebrand checklist starts with a full URL inventory rather than the pages the marketing team can name from memory.

The 15-minute hygiene routine

The fix does not require a project plan. It requires fifteen minutes, a calendar entry, and one named owner. In order:

1. Run the site: query. Search site:yourdomain.com in Google and read every result. You are looking for anything you would not put in front of a client: test pages, staging copies, duplicate drafts, expired campaign pages, internal tools. Note each offending URL.
2. Read your own sitemap.xml. Open yourdomain.com/sitemap.xml (or the sitemap index your SEO plugin generates) and scan the URL list. Anything there is being actively offered to the search engines and any other crawler that reads the file. If a URL should not be public, it should not be here – and its presence usually explains how it got indexed in the first place.
3. Check Google Search Console’s indexed-pages report. Under Indexing → Pages, review what Google actually holds – including URLs the site: query did not surface. This is also where you will spot the reverse problem: pages you want indexed that are not.
4. Remove properly: noindex, then verify. For each page that should vanish: unpublish it if nothing depends on it, or apply a noindex tag if it must stay live, then request recrawling via Search Console’s URL inspection tool – and check back until the page has actually left the index. Requesting a recrawl does not guarantee immediate removal; for urgent cases – a confidential document, a page that needs to be gone this week – use Search Console’s Removals tool to hide the URL temporarily while the permanent fix takes effect. Removal is complete when Google confirms it, not when you add the tag. And never “remove” anything via robots.txt – that preserves the problem while hiding the evidence.
5. Put staging behind authentication, not robots.txt. Your staging environment should require a login. A staging site is not private just because nothing links to it: its URLs get discovered through DNS history, referrer data, source code, analytics and the other trails a real site leaves. Password protection is the control; obscurity is not. A password prompt keeps every crawler out unconditionally; a robots.txt file politely asks them not to look, and only at the ones that ask first. If your staging site is reachable without credentials, this is the highest-value fix on the list.
6. Put a quarterly entry in someone’s calendar. Steps one to three, once a quarter, with a named owner. The leak is not a one-off event but a slow drip – the routine only works if it recurs. This check belongs alongside the rest of your website audit checklist as standing maintenance, not a special occasion.

WordPress checks to add

Most professional-services sites run on WordPress, and WordPress has leak points of its own. If that is your stack, add these to the routine:

• Attachment pages. WordPress can generate a thin standalone page for every uploaded image and PDF. Set your SEO plugin to redirect attachment pages to the file itself, or they multiply quietly in the index.
• Author, tag and thin taxonomy archives. A tag used once creates an archive page with one entry; on a small site, an author archive duplicates the entire blog. Noindex the archives you do not genuinely use as landing pages.
• SEO-plugin sitemap settings. Yoast, Rank Math and their peers decide which post types and taxonomies enter the sitemap. Open the settings and read the list – custom post types registered by themes and plugins often ride in unnoticed.
• Staging on subdomains. staging.yourfirm.co.uk is a real, resolvable host, not a hidden one. If it is not behind a password, assume it will be found.
• Old campaign landing pages. Pages built for a webinar, a seasonal offer or a PPC push rarely get a retirement date. Inventory them; most should be redirected or retired.
• Drafts published “temporarily”. Anything published to production for a quick test joins the sitemap within the hour. If it genuinely must go live, apply noindex on the way in, not afterwards.
• Canonicals after migrations. Cloned and migrated sites often carry canonical tags still pointing at the old domain or a staging URL. Spot-check the key templates after any move.
• The “discourage search engines” setting – on the right environment. Settings → Reading → “Discourage search engines from indexing this site” should be ticked on dev environments and unticked in production. We have seen both mistakes; the second is the expensive one.
• Test pages from themes, plugins and page builders. Themes and page builders create sample, demo and template pages on install. Search your page list for “sample”, “demo”, “template” and “test”, and deal with what you find.

Don’t make these three mistakes

The clean-up phase has its own traps, and we see each of these regularly:

1. Don’t use robots.txt to remove an already-indexed page. Blocking the URL stops Google recrawling it – which means Google can never see the noindex you added, so the page stays in the index indefinitely, stripped of its description. You have preserved the problem and hidden the evidence.
2. Don’t assume noindex removes the URL immediately. The tag only takes effect when the page is recrawled, and low-priority pages can wait weeks for that visit. Request recrawling via Search Console and check back until the page has genuinely gone.
3. Don’t delete URLs blindly. Even an embarrassing page may have accumulated backlinks, traffic or history worth keeping. Check what points at it before you decide how it should die.

Which leads to the actual decision rule, page by page: 301 redirect when a genuine replacement exists · 410 (or 404) when the page is genuinely gone and nothing should inherit its address · noindex when the page must stay accessible but out of the index · authentication, not robots.txt, for anything on staging.

Where this fits in the bigger picture

We have never audited a firm and found nothing with the site: query – including firms with excellent developers and diligent marketing teams. This is not a competence problem; it is an ownership problem, and it hides in plain sight because everyone assumes someone else is watching.

It is also why indexed-content hygiene is a standard line in the Intelligence Report of our SEO audit: what Google and the AI engines hold about your firm, page by page, against what you intended them to hold. In practice that means a full URL inventory, indexation findings, redirect recommendations and a prioritised remediation plan – the fifteen-minute routine above, done exhaustively and written down. If the fifteen-minute routine leaves you uneasy – or you would rather be told what is leaking before your prospects’ AI assistants find out – that is the conversation the audit is built to start.

Until then: site:yourdomain.com. Ten minutes, no tools, and you will know more about how machines see your firm than most of your competitors know about theirs.