Inside GONE: how I prove removed pages are really gone

Short answer: deciding what to do with a removed page is the easy part. Proving the server does it is where most clean-ups quietly fail. GONE takes your list of URLs and what you meant to happen to each one, then checks what really comes back. Every row is marked MATCH, WARN or MISMATCH, with a severity and a reason in plain words.

If you want the theory first (404 vs 410 vs soft 404), read 404 vs 410 vs soft 404. This piece is the maker's tour: what the code checks, the thresholds, the edge cases, and why it matters for money.

What you paste in

A CSV with three columns: url,intent,target. The intent is the decision you made for that URL:

• 410 or 404: the page is gone on purpose.
• 301 or 308: it moves, and target is where it should land.
• noindex: it stays live at 200 but leaves the index.
• unavailable_after: an event or offer page that expires, with the end time in target.
• 200: it must stay live and indexable.

Leave the intent blank and the mode's default fills it in: 410 in standard mode, 404 in facet mode, unavailable_after in post-event mode. GONE reads up to 2,000 rows a run. Add a sitemap, and it also flags any dead URL that is still listed.

The verdict rules, under the hood

The browser check and the downloadable gone.py script use the same rules, so a quick check and a full script report never disagree on what counts as a fail.

• Gone means gone at its own URL. For a 410 or 404 intent, a first response of exactly the code you chose is a MATCH. If you get the other one (404 when you meant 410), that's a Low warning, because Google treats them almost the same. A redirect is a High mismatch. A redirect to the home page gets its own line, because that can be treated as a soft 404.
• A 200 on a removed page is the expensive one. It's a High mismatch with a soft 404 warning. If the page also says noindex, GONE adds a note that noindex doesn't save crawl: Google still requests the page.
• Redirects have to be permanent and have to land. For a 301 or 308 intent: no redirect at all is High. A first hop of 302 or 307 is Medium (a weak signal). Landing somewhere other than your target is High. A target that returns anything but 200, or carries noindex, is also High.
• Hop count is measured. More than 3 hops is a warning, and 5 or more is High. That's in line with Google's advice to keep chains short.
• robots.txt can hide the answer. If a gone or noindex URL is blocked in robots.txt, Google never sees the 404, 410 or noindex. That's High.
• Sitemaps should only list live pages. A dead or non-indexable URL still in the sitemap is High. In migration mode it drops to a Low note, because during a move that's expected.

Each row's severity is the worst issue it has. Mismatches are then grouped into fix groups such as "Still returns 200", "Redirects instead of gone (home page)" and "Wrong redirect target". That way you hand developers one ticket per pattern, not 400 rows.

Edge cases I built for

• Faceted navigation. In facet mode, an empty filter page must return 404 at its own URL. A redirect away is High, and a 200 is High with a soft 404 warning.
• Events that end. A past-event page with no unavailable_after is Medium in post-event mode and High otherwise. GONE gives you the exact meta tag and the X-Robots-Tag header line to add.
• The browser can't see everything. In the browser, a redirect's first hop code is hidden, so GONE says so ("hop code not visible") instead of guessing. If a site blocks cross-origin requests, the row is BLOCKED, not failed. For both, run gone.py and import its report.

Worked example

url,intent,target
https://samplebrand.test/spring-offer,410,
https://samplebrand.test/old-guide,301,https://samplebrand.test/guides/new-guide
https://samplebrand.test/filter/colour-teal,,

Live response, row 1: 200 OK, still listed in sitemap.xml
Live response, row 2: 302 -> https://samplebrand.test/ (200)

Row 1 fails twice, as "returns 200, intended 410 - soft 404 risk" and as "still listed in sitemap". Row 2 fails on the 302 (Medium) and on landing at the home page instead of the target (High). Row 3 picked up the mode's default intent. That gives developers three ticket groups: serve the 410, fix the redirect rule to a permanent hop to the right target, and take dead URLs out of the sitemap.

Run it: open GONE and press Sample input

The money angle

Two costs hide in a botched removal:

• Crawl. Every soft 404 and long chain is a request Googlebot spends on nothing.
• Users. A redirect to the wrong place drops people on a page that doesn't answer them, and drops the link value you meant to pass on.

GONE puts a value on that. Each fix group carries a worst, base and best monthly value from your own assumptions (pages, clicks per page, uplift, conversion and value). The groups are ranked by the base case, and one click sends a group to PAYBACK to compare against the cost of the fix. The defaults are starting guesses, not benchmarks. Change them to your numbers before you show anyone.

This is the same proof I want before a reconsideration request, or after a migration: decision, then evidence. If you're mid-migration, the SEO migration work uses this exact loop.