Why Your Server Never Returns 304 to Googlebot

Open Search Console, go to Settings → Crawl stats → By response. For most sites, "Not modified (304)" is a thin sliver or missing entirely. Nearly every request Googlebot makes gets a full 200 with a complete HTML body, even for pages that have not changed in two years.

That is not Googlebot's choice. Googlebot supports conditional requests and Google has repeatedly encouraged sites to support HTTP caching properly — the Search Central "Crawling December" series in 2024 was explicit about it. The reason you never see 304s is almost always that your stack makes a valid conditional request impossible.

What a conditional request actually is

When Googlebot fetches a URL, it stores the validators your server returned: the ETag header, the Last-Modified header, or both. On a later crawl of that same URL, it can send them back:

GET /products/widget HTTP/1.1
Host: example.com
If-None-Match: "a3f91c-5e2"
If-Modified-Since: Tue, 04 Mar 2025 11:22:09 GMT

Your server compares. If the current representation still matches, it returns 304 Not Modified with headers and no body. If it does not match, it returns 200 with the full body as usual.

Two details matter. First, per the HTTP spec, when both headers are present the entity tag wins — If-None-Match is evaluated and If-Modified-Since is ignored. A broken ETag therefore defeats a perfectly good Last-Modified. Second, you cannot force Googlebot to send a conditional request. It decides, based on what it has cached and its own scheduling. You can only make sure that when it does, your server answers correctly.

What a 304 saves, and what it does not

A 304 does not save a request. The TCP connection, the TLS handshake, the DNS lookup, the log line — all still happen. What it saves is body generation and body transfer.

For a typical HTML page that means skipping the template render, the database queries behind it, and 60–150 KB of transfer. That is the entire benefit, and it is a real one, because Google has documented that crawl rate responds to host load: when a host is slow or returns server errors, Googlebot backs off; when responses are fast and clean, it is willing to ask for more. A fleet that answers re-crawls with cheap 304s is a faster host.

What a 304 does not do:

• It is not a ranking signal and not a freshness signal you can game.
• It does not refresh or reset anything about the indexed version. Google reuses what it already has.
• It does not map cleanly to "crawl budget" as a request quota. If you have a crawl budget problem driven by millions of junk URLs, caching headers will not fix it — that is a URL-space problem.

The reasonable inference, not documented behavior, is that a URL that answers 304 over and over teaches the scheduler it changes rarely and gets re-crawled less aggressively. Treat that as plausible, not certain.

Why your server never sends one

In rough order of how often I see it:

The page has no validators at all. Most application frameworks emit neither ETag nor Last-Modified for dynamic HTML, and many add Cache-Control: no-store or private on top. With nothing cached, Googlebot has nothing to send back.

The ETag changes on every render. Middleware that hashes the full response body will produce a new ETag whenever anything in the HTML varies: a CSRF token, a nonce in a CSP header reflected inline, a rendered timestamp, a rotating testimonial, a cache-buster query string on an asset. The page looks identical to a human and is byte-different on every request.

The ETag differs between servers. Apache's FileETag has historically been derived partly from the file's inode number, which differs across machines in a load-balanced fleet. The same file then has a different ETag depending on which backend answers. Apache 2.4 changed the default, but hardened configs and inherited .htaccess files still set INode MTime Size.

Compression mangles it. mod_deflate historically appended -gzip to the ETag, so a client that cached the uncompressed tag never matched. nginx's handling of ETags with on-the-fly gzip has changed across versions. Verify your actual behavior with Accept-Encoding: gzip set; do not assume.

Last-Modified is the deploy time, not the content time. A static site generator that rewrites every file on every build gives you a fresh timestamp for 40,000 unchanged pages. Technically valid, practically useless.

The server never implements the comparison. Plenty of app servers emit an ETag and then ignore If-None-Match entirely, returning 200 every time. Emitting a validator is half the job.

A CDN or WAF strips or rewrites the headers, or the origin sets a Vary that the edge does not reproduce, so the conditional never reaches logic that can satisfy it.

How to test this in under a minute

URL=https://example.com/some-page
# 1. What validators does the page emit?
curl -sI -H 'Accept-Encoding: gzip' "$URL" \
  | grep -iE 'etag|last-modified|cache-control|vary'

# 2. Does the server honor them?
ETAG=$(curl -sI -H 'Accept-Encoding: gzip' "$URL" | awk -F': ' '/[Ee]Tag/{print $2}' | tr -d '\r')
curl -s -o /dev/null -w '%{http_code} %{size_download}\n' \
  -H 'Accept-Encoding: gzip' -H "If-None-Match: $ETAG" "$URL"

# 3. Is the ETag stable across repeat fetches and across backends?
for i in 1 2 3; do curl -sI "$URL" | awk -F': ' '/[Ee]Tag/{print $2}'; done

Step 2 should print 304 0. Step 3 should print the same value three times. Run the whole thing again with -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)", because edge rules and bot-management layers frequently serve crawlers a different path.

Reading the diagnosis from the symptoms

In server logs, segment by verified Googlebot and compute the share of 2xx/304 responses and bytes sent per directory. A 0% 304 rate on a 200,000-page catalog that changes monthly is a quantifiable waste of origin capacity. In Search Console, the Crawl stats "By response" breakdown gives you the same ratio from Google's side without log access, though at lower resolution.

The failure mode that is worse than no 304

Serving a 304 for content that did change is far more damaging than never serving one. Googlebot keeps the version it has. If you edited the page, added a noindex, changed the canonical, or fixed a price in structured data, none of it lands until the validator finally changes.

So the rule is strict: the validator must change whenever the rendered output changes. That includes the head, the structured data block, and anything injected server-side. If you cannot guarantee that coupling — for instance on a page assembled from several independently cached fragments — emit no ETag at all and let Googlebot take the 200.

Also resist the temptation to use Cache-Control: max-age as a re-crawl dial. Google has not documented a direct mapping from max-age to crawl interval, and the scheduler has its own inputs. Treat it as a hint, at most.

What to do

1. Test ten representative URLs — homepage, category, product/article, paginated page — with the curl sequence above, under both a browser and a Googlebot user agent.
2. For any page type with an unstable ETag, find the varying bytes by diffing two responses. Strip per-request noise out of the cacheable body, or move it client-side.
3. If you cannot stabilize the ETag, drop it and ship a correct Last-Modified tied to content modification time, not build time. One good validator beats two broken ones.
4. Confirm the origin actually evaluates conditionals. Many stacks need this turned on explicitly.
5. Check that your CDN forwards If-None-Match and If-Modified-Since to the origin and passes 304s back unmodified.
6. Record today's 304 share from Crawl stats, make the change, and compare after four to six weeks. Also track bytes served to verified Googlebot in your logs — that number moves first and is less noisy.

The realistic outcome is a cheaper, faster origin for the slice of crawl traffic that re-visits stable URLs. Whether that converts into more crawling of the URLs you care about depends on your site, and is worth measuring rather than assuming.