SEO Regression Testing: Catch What a Deploy Breaks
The SEO regressions a deploy causes most often (noindex, robots.txt, canonicals, titles, redirects), the checks to run in CI and after release, and how to tell a real drop from noise.
SEO regression testing means checking, on every release, that the deploy did not break what Google reads: whether a page may be indexed, which URL is canonical, what the title says, where old URLs go, and which pages link where. These breaks are quiet. The page still loads, every product test passes, and the drop shows up in Search Console a week or two later, when the cause is several deploys back.
TL;DR:
- What breaks: a
noindexcarried over from staging, aDisallowin robots.txt, a canonical pointing at the wrong URL, a template that empties or duplicates titles, removed redirects, and links that disappear from the HTML. - Before release: fetch your key URLs in CI and assert on the response: status, robots rules, canonical, title, redirects.
- After release: run URL Inspection on the pages that matter, and watch Page indexing and Crawl stats for a spike.
- When traffic drops: compare the page against the rest of the site over the same days before you blame the deploy.
What does a deploy break most often?
Five things, all of them invisible to a visitor:
- A
noindexthat reached production. Staging environments are usually markednoindex, and the rule can ship with the release. It lives in two places: a<meta name="robots" content="noindex">tag, or "anX-Robots-TagHTTP header with a value of eithernoindexornone". A header check is easy to forget because the HTML looks clean. - A robots.txt that blocks crawling. A staging
Disallow: /copied over stops Googlebot fetching pages. It also hides a fix: "For thenoindexrule to be effective, the page or resource must not be blocked by a robots.txt file." Google adds that robots.txt "is not a mechanism for keeping a web page out of Google", so a blocked URL can stay in results with no description. - A canonical that points somewhere else. Google treats
rel="canonical"as "A strong signal that the specified URL should become canonical." A template bug that points every page's canonical at the homepage, or at a staging host, signals that the page is a duplicate of that URL. - Titles emptied or duplicated. A layout change that drops the per-page
<title>leaves Google to "generate an improved title link from anchors, on-page text, or other sources." That is the line people click in results. - URLs that changed without redirects. A renamed route returns 404 where it used to rank. Google calls a redirect "A strong signal that the target of the redirect should become canonical", which is why the old URL needs one. Even with redirects, "If you change the URLs of existing pages on your site, you may experience ranking fluctuations."
A sixth one is slower to notice: internal links that were in the HTML and now render only after JavaScript, or were removed with a navigation redesign. The page still ranks, but the pages it used to link to lose a path in.
How do you test for SEO regressions before a release?
Treat it like any other regression suite: a list of key URLs and an assertion for each rule. Pick the pages that earn search traffic, one of each template, and every URL that changed recently. Then check the response a crawler gets, not the page a browser renders after scripts run.
A minimal version in shell, run against a preview deploy:
BASE="https://preview.example.com"
failed=0
fail() { echo "FAIL $1"; failed=1; }
for path in / /pricing /blog/some-post; do
url="$BASE$path"
headers=$(curl -sI "$url")
body=$(curl -s "$url")
echo "$headers" | grep -q "^HTTP/[0-9.]* 200" || fail "status: $path"
echo "$headers" | grep -qi "x-robots-tag:.*noindex" && fail "header noindex: $path"
echo "$body" | grep -qi '<meta name="robots"[^>]*noindex' && fail "meta noindex: $path"
echo "$body" | grep -q "<link rel=\"canonical\" href=\"https://www.example.com$path\"" || fail "canonical: $path"
echo "$body" | grep -q "<title>[^<]\+</title>" || fail "empty title: $path"
done
curl -s "$BASE/robots.txt" | grep -qx "Disallow: /" && fail "robots.txt blocks everything"
exit $failed
Grow it the same way you grow unit tests. Every time a regression reaches production, add the assertion that would have caught it. Three more are worth adding early:
- Old URLs redirect. Keep a list of retired paths and assert each returns 301 to the right target, not 200 or 404.
- Titles are unique. Collect titles across the key URLs and fail on duplicates.
- The sitemap matches. Every key URL is in the sitemap, and nothing in the sitemap returns a non-200 status.
Run these on the preview build in CI and again against production right after deploy. The production run catches what only exists there: CDN rules, environment variables, a robots.txt served by another service.
Try it
Point an SEO agent at your own site, not just an essay about one.
QueryScope holds the plan for your site, reads your real Search Console, and hands the coding agent you already have one change at a time to rank for the searches you decided to win. From $49/month, everything included.
What should you check after the release?
What only Google can show you:
- URL Inspection on a few key pages, with a live test. It shows whether the page is indexed, which canonical Google chose, and whether crawling was allowed; the live test checks the version you just shipped. How to read URL Inspection goes through each field.
- Page indexing and Crawl stats. Google's guide to traffic drops says to check both "to find if there's a corresponding spike in issues detected" when the cause may be technical. A jump in pages excluded by
noindexor blocked by robots.txt right after a deploy dates the regression for you. If a page has already fallen out, why a page dropped from Google's index covers getting it back. - Performance, a week or two later. Google has to recrawl before anything moves, so the effect of a deploy shows up late. How long Google takes to recrawl a page covers the wait.
Was it the deploy, or just noise?
Rankings move every day without anyone touching the site, so a lower week after a release proves little on its own. Three checks before you roll anything back:
- Is it one page or the whole site? Google's guide separates issues that are "site-wide (for example, your website is down) or page-wide". A drop on every page points to an update, a seasonal dip or a site-wide break. A drop on only the pages the deploy touched points at the deploy.
- What did the rest of the site do over the same days? If every page fell 15%, a page that fell 15% did not lose anything of its own. Why Google rankings fluctuate covers how far rankings move with nobody touching the page.
- Was a Google update running? Compare the dates. How to tell if a Google update hit your site walks through it.
Regression testing and split testing answer different questions. A regression suite checks that nothing broke; an SEO A/B test checks whether a change helped, and it needs far more traffic. SEO A/B testing covers that side.
Keeping a record of every SEO change
The hard part of placing a drop is knowing what shipped, on which page, on which day. Commit history has the code but not what each change was meant to do in search. QueryScope keeps that record for teams whose site is a codebase. Your coding agent records each change as it ships it: the page, what it touched (the title, the copy, or only what a crawler sees), and what it expects to move. The change is then read against your stored Search Console history from the day Google rereads the page, beside what the rest of the site did where there is enough traffic to compare. An edit made outside that loop, like a template fix, is recorded too, so a later reading on that page knows it happened.
Sources
- Google Search Central, Block Search indexing with noindex (
noindexvia a<meta>tag or "anX-Robots-TagHTTP header with a value of eithernoindexornone"; "For thenoindexrule to be effective, the page or resource must not be blocked by a robots.txt file"). - Google Search Central, Introduction to robots.txt (robots.txt "is not a mechanism for keeping a web page out of Google"; a blocked page's URL "can still appear in search results").
- Google Search Central, How to specify a canonical URL with rel="canonical" and other methods (redirects: "A strong signal that the target of the redirect should become canonical";
rel="canonical": "A strong signal that the specified URL should become canonical"). - Google Search Central, Influencing your title links in search results ("Make sure every page on your site has a title specified in the
<title>element"; Google "may try to generate an improved title link from anchors, on-page text, or other sources"). - Google Search Central, Debugging drops in Google Search traffic (issues "can be site-wide (for example, your website is down) or page-wide"; "Check the Crawl stats report and Page indexing report to find if there's a corresponding spike in issues detected"; "If you change the URLs of existing pages on your site, you may experience ranking fluctuations").
Try it
Put an SEO agent in your editor.
Ask your coding agent what to work on. QueryScope reads your real Search Console, holds the searches you decided to win, and hands back one move at a time with the proof it worked. From $49/month, everything included. 1, 3 or 10 sites.