Structured data has a failure mode most markup does not: it breaks invisibly. A template refactor drops the offers block from product schema, every page still renders perfectly, every test passes, and the first signal anyone sees is rich results quietly disappearing and a warning trickling into Search Console weeks later.
That lag is the whole problem. Search Console tells you about regressions after Google has recrawled enough pages to notice. A CI check tells you before the deploy ships. If your schema is generated by templates — and it almost always is — it should be tested like the code it is.
Why structured data regresses
Almost nobody hand-writes JSON-LD per page. It is assembled by a template, a CMS plugin, or a component from the same data that feeds the visible page. That creates predictable breakage paths:
- A refactor renames or removes a field the schema template reads, and the property silently emits as empty or vanishes.
- A CMS or plugin update changes its schema output without anyone reviewing what it now emits.
- Client-side rendering shifts so the JSON-LD that existed in server HTML now appears only after hydration, or twice — once from the server, once from a component.
- Escaping bugs produce JSON that no longer parses at all — one unescaped quote in a product name is enough.
None of these throw errors, because nothing consumes the markup at build time. The fix is to make something consume it.
The two layers of validation, and which tools cover which
Validation is really two different questions, and conflating them causes confusion.
Syntactic and vocabulary validity: does the JSON-LD parse, and does it use schema.org types and properties correctly? The Schema.org validator (validator.schema.org) answers this. It knows nothing about Google.
Rich result eligibility: does the markup meet Google's documented requirements for a specific rich result — the required and recommended properties listed in Google's search gallery documentation? Google's Rich Results Test answers this.
On automating the second layer, be precise about what exists. Google does not currently document a standalone public API for the Rich Results Test the way it once did for AMP testing. The supported programmatic route is the Search Console URL Inspection API, which returns a richResultsResult verdict — but it inspects URLs on a verified property, which makes it a post-deploy monitoring tool, not a pre-deploy gate. Some third-party validators wrap Google's documented requirements into libraries you can run anywhere; that works, with the caveat that the requirements are maintained by the third party, not Google.
The practical consequence: your CI gate is built from parsing plus your own assertions, and the Google-verdict layer runs against production on a schedule.
What a CI check should actually assert
The highest-value check is not generic validation. It is asserting that each template type emits the schema you decided it should emit. Generic validators cannot know that your product pages are supposed to have offers.price — only you know that. So the pipeline looks like:
- Build the site, or spin up a preview environment.
- Fetch a fixture set of representative URLs — one or two per template: product, article, recipe, event, whatever you ship.
- Extract every
<script type="application/ld+json">block from the rendered HTML. - Parse each block. A parse failure fails the build.
- Assert per-template expectations: required types present, required properties present and non-empty, values sane.
A minimal extractor is genuinely small:
import json, sys
from html.parser import HTMLParser
class LdJson(HTMLParser):
def __init__(self):
super().__init__(); self.grab = False; self.blocks = []
def handle_starttag(self, tag, attrs):
self.grab = tag == "script" and ("type", "application/ld+json") in attrs
def handle_data(self, data):
if self.grab:
self.blocks.append(json.loads(data)) # raises on broken JSON
p = LdJson(); p.feed(sys.stdin.read())
types = {b.get("@type") for b in p.blocks}
assert "Product" in types, "product template lost its Product schema"The assertions are the part worth investing in. Encode Google's documented required properties for each rich result type you rely on, plus your own rules — price is numeric, image URLs are absolute, datePublished is a real date. When a template refactor drops a field, this fails the pull request instead of a rich result.
Test the HTML that crawlers get, not the source
If your site renders client-side, extracting JSON-LD from raw server HTML tests the wrong artifact — and extracting it from a headless browser tests a different one. Google has documented that it can process JavaScript-inserted structured data, but rendering is deferred and adds a dependency you do not need. The robust position is schema in the initial server response. Whatever you choose, make CI fetch pages the same way you expect crawlers to receive them, and consider asserting the markup exists pre-rendering so a hydration-only regression gets caught.
What automation cannot promise
Two boundaries to keep explicit. First, valid markup is a precondition for rich results, not a guarantee — Google has documented that eligibility does not ensure display. CI can prove you did not break your side; it cannot promise the outcome. Second, some Search Console structured data issues stem from mismatches between markup and visible content, which no parser can judge. Automation removes the mechanical regressions, which in practice is where most of them come from.
What to do
- Inventory the schema you actually rely on — the types tied to rich results you care about — and write down required properties per template.
- Add the CI gate: fixture URLs per template, extract JSON-LD from rendered output, fail the build on parse errors or missing required properties.
- Run Google's verdict on a schedule, not in CI: weekly URL Inspection API checks (or manual Rich Results Test spot checks) against production fixtures, alerting on verdict changes.
- Treat Search Console's structured data reports as the trailing indicator they are — confirmation that the gate is working, not the gate itself.
- Re-review after every CMS or plugin upgrade that touches schema output, because that is the regression path your tests do not sit in front of.