Pull your domain's backlink profile from three tools and you will get three different referring-domain counts, often differing by a factor of two or more. The usual reaction is to decide which tool is "most accurate" and standardize on it. That framing is wrong, and it produces bad decisions downstream — most often, panic about "lost links" that were never lost.
A backlink index is not a registry of the web's links. It is one company's crawl, filtered by one company's rules about what counts as a link, a domain, and a live page. Two indexes disagreeing tells you about their crawl coverage and counting conventions, not about reality. Once you know which disagreements are structural, you know which numbers to act on.
What a backlink index actually is
Every commercial link index is built the same basic way: a crawler fetches HTML, extracts <a href> elements, and writes edges into a graph. There is no privileged feed from Google. The index is bounded by three things:
- Seed set and crawl frontier. The crawler can only find links on pages it fetches, and it fetches pages it has discovered from other pages. Isolated corners of the web — a niche forum nobody links to, a regional directory — may simply never enter the frontier.
- Crawl budget allocation. Vendors prioritize by their own quality and popularity heuristics. A link from a high-authority news site gets discovered fast. A link from a page on an unpopular domain, five clicks deep, may wait months or never be crawled.
- Retention and pruning rules. Indexes drop spam, parked domains, and pages that stopped responding. Different pruning thresholds produce different totals even on identical crawl data.
Majestic has historically been explicit about this, splitting its data into a "Fresh" index of recently crawled pages and a "Historic" index covering a much longer window. Most vendors blend these concepts without surfacing the seam, which is a large part of why counts feel arbitrary.
The counting rules that create most of the gap
Before any crawl difference, vendors disagree on definitions. These decisions compound:
| Decision | Common variants | Effect on your count |
|---|---|---|
| What is a "referring domain" | Registrable domain vs. hostname | Subdomain-heavy link profiles inflate under hostname counting |
| Sitewide links | Count every URL vs. collapse to one per domain | Footer/blogroll links swing total backlinks by orders of magnitude |
nofollow, ugc, sponsored | Included in totals, excluded, or reported separately | Forum and comment-heavy profiles shift hugely |
| Links through redirects | Attributed to the final target, the redirect URL, or both | Migrated sites look very different between tools |
| Links in rendered DOM only | Counted or ignored | JS-injected links missing from raw HTML indexes |
| "Lost" link criteria | One failed fetch vs. repeated confirmation | Drives false lost-link alerts |
None of these is wrong. They are modeling choices. But it means a sentence like "we have 4,200 referring domains" has no meaning outside the tool that produced it.
Why links appear late and disappear later
The timing asymmetry is the part most people miss. Discovery is cheap; confirmation of loss is expensive.
To add a link, the crawler needs one successful fetch of one page. To confirm a link is gone, it has to re-fetch the source page, parse it, and see the anchor absent — and ideally do that more than once, because a single 503 or a rendering failure is not proof of removal. Vendors that mark links lost aggressively produce noisy churn charts. Vendors that confirm conservatively show links that have been dead for weeks.
The practical consequence: a drop in your "live" link count in any tool is at least as likely to reflect that tool's recrawl schedule as an actual editorial removal. Before you email anyone about a pulled link, fetch the source URL yourself.
What JavaScript and redirects do to link discovery
Two mechanisms create systematic blind spots rather than random noise.
Client-side rendering. If a link only exists after JavaScript executes, an index that parses raw HTML will never see it. Vendors that render do so selectively, because headless rendering is far more expensive per page than fetching HTML. Google has documented that its own rendering is deferred and queued; third-party crawlers face the same economics with much smaller budgets. The reasonable inference is that links on JS-heavy platforms — some SPAs, certain comment and community widgets — are structurally underrepresented in every commercial index.
Redirect chains. When a linking page points at a URL that now 301s, attribution depends on whether the vendor follows and re-attributes. If you migrated domains, one tool may show your links against the old host, another against the new one, and a third split across both. This is also why post-migration link reports look alarming for months. Check whether the tool has a setting for including redirected links before concluding anything.
Which link numbers are actually usable
Treat absolute counts as unusable and derived comparisons as usable, with conditions:
- Deltas within one tool, one definition, one date range. Your referring domains in Tool A this month vs. last month is a real signal, assuming the tool did not change its index or counting rules — which does happen, usually announced.
- Intersections across tools. A domain present in two independent indexes is almost certainly real. Union sets are better for discovery; intersections are better for confidence.
- Verified-live subsets. Export the referring URLs, fetch them yourself, and check that the anchor exists in both raw HTML and rendered DOM. This is the only count you can defend.
- Relative comparisons against competitors in the same tool. Coverage bias applies roughly equally across sites in the same vertical, so rank-order comparisons survive better than absolute ones.
What does not survive: using one vendor's domain-level authority score as a threshold in outreach, then reporting the resulting average back as progress. That is measuring the tool's model, not the web.
How to build a link dataset you can defend
- Export from at least two indexes, with
nofollowand redirected links included, then filter yourself. You cannot recover data a tool excluded. - Normalize to one key before joining. Something like
registrable_domain + "|" + normalized_target_path, lowercased, query string stripped unless it is meaningful. - Store a
first_seenandlast_verifiedtimestamp per edge from your own verification fetches, not the vendor's. Your churn chart should be built on your fetches. - Re-verify any "lost" link at least twice, several days apart, before treating it as removed. Record the HTTP status; a 403 to your fetcher is not a removal.
- Freeze your definitions in writing and put them in the footer of every report. When someone asks why the number differs from the tool's dashboard, the answer should be a document, not a guess.
The goal is not a number that matches someone else's number. It is a number whose construction you can explain, measured the same way every month.