We moved voxtrend.fr from Webflow to Astro. The redirects were written, the build was green, the site was live. Two months later I went looking for something else and found eleven URLs still sitting in Google's index, all returning 404.
None of them were in the redirect map. Not because anyone was careless : because nobody remembered they existed.
The failure mode
When you migrate, you write redirects for the URLs you know about. You get them from the old CMS export, or from the sitemap, or from memory.
That covers the pages you published. It misses:
pages you deleted a year before the migration, which Google still has
demo content from the CMS template you never cleaned up
URL variants the old CMS generated on its own
pages in a language or section you dropped along the way
Every one of those may still hold external links. A 404 throws that away.
The Wayback Machine has the list
The Internet Archive exposes a CDX API that returns every URL it ever captured for a domain. No key, no account, no rate limit worth worrying about for a one-off.
curl -s "https://web.archive.org/cdx/search/cdx?url=voxtrend.fr&matchType=domain&output=json&fl=original&collapse=urlkey&filter=statuscode:200&limit=500" -o cdx.json
The parameters that matter:
matchType=domain — the whole domain, not just the exact URL
collapse=urlkey — one row per URL instead of one per capture
filter=statuscode:200 — only pages that actually resolved
fl=original — return just the URL column
Then reduce it to unique paths and drop the assets:
import json, re, urllib.parse
rows = json.load(open('cdx.json'))[1:] # first row is the header
paths = set()
for (url,) in rows:
p = urllib.parse.urlparse(url).path.rstrip('/') or '/'
if re.search(r'.(png|jpe?g|svg|css|js|woff2?|ico|webp|pdf)$', p, re.I):
continue
paths.add(p)
for p in sorted(paths):
print(p)
What that turned up
Fifty-two historical paths. Most were already redirected. Eleven were not, and they fell into two very different groups.
Real pages, still linked from outside:
/about-us
/politique-de-confidentialite
/privacy-policy
/blog-2
/connexion
/creation_de_compte
Leftovers from the Webflow starter template:
/blog-post/best-prototyping-tools-12
/blog-post/best-prototyping-tools-13
/career-post/interface-designer-2
/integrations-post/blossom
/blog-categories/banking
Someone had launched from a template and never deleted the demo content. It sat there long enough to get crawled.
Check which ones are still live before you write anything:
for p in $(cat paths.txt); do
printf "%-44s " "$p"
curl -s -o /dev/null -w "%{http_code}\n" "https://voxtrend.fr$p"
done
Redirect the first group. Let the second one 404.
This is the part people get wrong, and it's counter-intuitive.
For the real pages, redirect to the closest equivalent. /about-us to the homepage if there's no about page anymore. /privacy-policy to the French version. Those pages existed, people linked to them, the link should land somewhere meaningful.
For the template junk, do nothing. Let it 404.
Bulk-redirecting a hundred dead URLs to your homepage is a documented soft-404 pattern. Google treats a redirect to an irrelevant page the same as a 404, and you've added a hop for nothing. A page that never had real content has nothing to preserve. A clean 404 is the correct answer.
Two things I checked while I was in there
Vercel's permanent: true emits a 308, not a 301. If your SEO tool flags "308 instead of 301", ignore it : Google treats them identically. Both are permanent, both pass signals. The difference is that 308 preserves the HTTP method, which is irrelevant for page redirects.
The canonical host was inconsistent. astro.config.mjs declared site: 'https://voxtrend.fr' while every page and the sitemap used https://www.voxtrend.fr. Nothing was broken, because nothing read Astro.site yet. The day you add @astrojs/sitemap or an RSS feed, they emit non-www URLs that all bounce through a redirect. Worth fixing before it becomes a real bug.
The whole thing takes twenty minutes
One API call, one script, one loop of status checks. It found six real URLs that had been quietly discarding whatever external links pointed at them.
If you migrated a site in the last year and only redirected what was in the CMS export, run the CDX query. The list is longer than you think.
