Modeling Internal PageRank From Your Own Crawl

Most internal linking audits stop at click depth: pages three clicks from the homepage are fine, pages six clicks deep are "orphaned-ish," go add some links. Click depth is easy to compute and it correlates loosely with what you care about, but it throws away almost all the information in your link graph. A page one click from the homepage via a single footer link is not equivalent to a page one click away from a heavily-linked hub.

The better model is PageRank computed over your own site's internal links. It is a twenty-five-year-old algorithm, it is not what Google runs today, and it is still the most useful single number you can attach to an internal page. Here is how to build it correctly and how to read it without over-trusting it.

What PageRank actually computes

PageRank models a random surfer who starts on some page and repeatedly does one of two things: with probability α (conventionally 0.85, the damping factor), follow a random outbound link on the current page; with probability 1−α, teleport to a random page in the graph. PageRank is the long-run share of time the surfer spends on each page.

Two consequences matter for internal linking:

• Value divides by out-degree. A page with 200 outbound links passes roughly a two-hundredth of its score down each edge. Mega-menus and "related posts" blocks with 40 entries dilute everything they touch.
• It is recursive, not distance-based. Being linked from a page that is itself well-linked is worth more than being linked from a leaf page. Click depth cannot express this.

Google representatives have said over the years that some form of PageRank remains part of the system, while also making clear it is not the algorithm from the 1998 paper and that the public Toolbar PageRank number was retired in 2016. Treat your computed values as a model of link flow on your site, not an estimate of anything inside Google.

How to build the edge list without garbage in it

The modeling is trivial. Getting the graph right is the whole job. Export the crawl from Screaming Frog, Sitebulb, or your own crawler and apply these rules before you build edges:

1. Resolve redirects. An edge to a URL that 301s to another URL should be rewritten to point at the destination. Otherwise you accumulate score on URLs that no user or crawler ever lands on.
2. Collapse to canonicals. If /product?color=red canonicalizes to /product, rewrite the edge target. Canonicalization is a hint rather than a directive, so where Google has chosen a different canonical than you declared — visible in the URL Inspection API — use Google's choice.
3. Drop non-200 targets and non-indexable targets. Links into noindex pages still consume out-degree from the source, so keep them as outbound edges when computing the source's out-degree, but don't rank them.
4. Deduplicate edges per source page. Ten links from the same article to the same target is one edge, not ten. Most crawler exports give you raw link instances.
5. Include only rendered links if your site needs JS. If navigation is client-side, a raw-HTML crawl will produce a graph that does not exist.
6. Handle nofollow by removing the edge, not redistributing it. Since 2009 Google has stated that PageRank sculpting does not work: the share allocated to a nofollowed link evaporates rather than flowing to the remaining links. Model that by keeping the link in the out-degree denominator but not adding the edge.

Then the computation itself:

import networkx as nx

G = nx.DiGraph()
G.add_edges_from(edges)  # canonicalized, deduped, internal only
pr = nx.pagerank(G, alpha=0.85)

Seeding the graph with where authority actually enters

Default PageRank teleports uniformly to every page, which implies every URL is an equally likely entry point. That is wrong for almost every site. Authority enters through pages that have external links, and traffic enters through pages that rank.

networkx accepts a personalization dictionary — a weighted teleport distribution. Two useful seedings:

• Referring domains per URL, from your backlink tool of choice. This approximates where external link equity lands before it flows internally.
• Organic entrances or impressions per URL, from Search Console. This models attention rather than authority, but it is often the more actionable of the two.

pr = nx.pagerank(G, alpha=0.85, personalization=ref_domains_by_url)

Run both. The pages that rank high on the link-seeded graph and low on the traffic-seeded graph are usually template artifacts — "About us," "Privacy" — hoarding flow they cannot use.

Reading the output: ratios, not absolute values

PageRank scores sum to 1 across the graph, so the raw numbers mean nothing on their own and are not comparable between crawls of different sizes. What you want is the rank of a page relative to its commercial or strategic importance. Join your PageRank output to revenue, conversions, or impressions per URL and look at the mismatches.

A useful derived metric is PageRank per unit of value: sort by score divided by (impressions + 1) and the top of that list is where link equity is being wasted.

Where this model is definitively wrong

Be honest about the gaps, because the failure mode of this analysis is over-confidence.

• It ignores anchor text and link position. Google has documented that link context and prominence matter; classical PageRank treats a footer link and an in-body editorial link identically. Some practitioners weight edges by position — reasonable, but it is inference, not documented behavior, and you are now tuning a parameter with no ground truth.
• It ignores topical relevance. A link from an unrelated page counts the same. Real systems almost certainly do not work this way.
• The damping factor is a convention. Results are fairly stable between 0.8 and 0.9; if your conclusions flip when you change it, they were not conclusions.
• It is a snapshot of a crawl, not of what Google has indexed. Pages Google has not discovered do not benefit from links Google has not crawled.

What to do with this

Run the crawl, build the edge list with the six cleaning rules above, compute PageRank twice — uniform and seeded with referring domains — and join both to Search Console impressions by URL. Then look only at the residuals: the twenty highest-value pages with the lowest scores, and the twenty highest-scoring pages with no commercial role.

Fix the second list first. Removing links is faster than adding them, has no editorial cost, and increases the share flowing to everything else on the same templates. Then re-crawl after the change ships and confirm the graph moved the way you predicted — if it did not, your edge list was wrong, not the algorithm.