How we cut internal link auditing from 100 hours to an hour

We turned an annual SEO chore into an on-demand command. And no, this doesn't mean we're replacing anyone at Apify with AI.

The problem: Who's got the time to audit 700+ blog posts?

The Apify blog has 738 published posts across 31 topic clusters. Like every long-running content operation, it has accumulated structural debt: orphan posts that nothing links to, pillar pages that don't link to their own strongest supporting content, anchors that say "click here" instead of what the user is actually clicking through to.

Internal linking is one of the strongest SEO levers we have. It tells search engines which posts are authoritative within a cluster, distributes link equity from popular pages to deep cuts, and gives readers a guided path through related content. When it's broken, every post pays a small tax.

The problem isn't that we don't know this. The problem is auditing it at scale, and doing it often enough that the findings stay current.

What a manual audit looks like

A thorough internal-link audit on 738 posts requires, per post:

Pull inbound internal links (Ahrefs or internal search): ~2 minutes
Review outbound internal links for anchor quality and topical fit: ~5–8 minutes
Classify the post's role in its topic cluster (pillar / supporting / standalone) and check that its pillar links are intact: ~2–3 minutes
Note required fixes in a tracker: ~2 minutes

That's 10–15 minutes per post for a single reviewer. For 738 posts:

Low estimate: 10 min × 738 = ~123 hours (~3 weeks of full-time work)
High estimate: 15 min × 738 = ~184 hours (~4.5 weeks)

Tools like Screaming Frog and Ahrefs Site Audit automate the crawl but don't replace the judgment; they just tell you a link exists, not whether the anchor is descriptive or the link is topically relevant. The synthesis work is still human.

Realistically, a competent SEO analyst can deliver a thorough audit of a blog this size in 3–4 weeks of focused work. Which is why most teams do it once a year, if ever.

How we solved the problem with Claude Code and an Apify Actor

We built a self-driving link auditor packaged as a Claude Code skill. It runs on a custom Apify Actor plus a small LLM pipeline and exposes a conversational front-end through Claude.

The pipeline (designed to complete a full pass on 738 posts in 30–60 minutes; LLM analysis dominates the runtime):

Scrape every blog post.
Build the link graph — every post → post edge, with anchor text.
Rule-based signals — flag generic anchors, count orphans, detect over-linking.
Cluster detection — group posts by primary tag, identify a pillar per cluster (highest intra-tag inbound).
LLM verdicts — for every internal link, ask the model two questions: is the anchor descriptive? Is the target topically relevant?
Score — 0–100 per post combining all signals.
Reports — six CSV/JSON files: per-post fix lists, cluster gaps ranked by leverage, orphan list, raw link graph.

The conversational layer

The skill teaches Claude how to interpret the reports. An editor types a question in plain English, and Claude reads the relevant CSVs, picks the right data, and writes a recommendation grounded in the actual numbers.

What changed in v2: expanding beyond the blog

The first version of the auditor only looked at links within blog.apify.com. That answered the question "which blog posts link to which other blog posts?", but it overstated the orphan issue. A blog post with zero inbound links from other blog posts might still be linked heavily from docs.apify.com, an apify.com landing page, or Apify Store Actor pages. By the blog-only definition, it was an orphan. In reality, it wasn't.

v2 added an includeExternalApifySources flag. When enabled, the Actor also crawls docs, the marketing site, and Apify Store (~230,000 pages) and folds every inbound link to a blog post into the graph. The result is a true cross-property view: a post only counts as an orphan if nothing across all of Apify's properties links to it.

The tradeoff is runtime. The blog-only pass finishes in 30–60 minutes. The full-scope pass crosses Claude Desktop's background-step timeout, so the skill is now designed to be run from Claude Code in the terminal, which has no such limit. The skill auto-detects the global install at ~/.claude/skills/link-audit/, so it works from any directory.

The receipts from one run

The most recent (blog-only) audit produced the following in a single pass:

738 posts analyzed
~13,000 internal links examined
31 topic clusters detected
302 orphans identified (~40% of the blog had zero inbound links from other blog posts)
996 cluster gaps ranked by leverage
608 posts flagged with at least one concrete fix

The structural findings above (orphans, gaps, scores) are produced even without the LLM phase. The LLM phase adds anchor-quality and topical-relevance verdicts.

Sample finding: the Python cluster

In the full-scope pass, the most significant missing link in the entire blog was the edge between our Python web scraping tutorial (the cluster pillar) and The 9 best Python web scraping libraries (priority score 34,407). The pillar wasn't linking out to one of the most authoritative supporting pages in its own cluster. Sitting right next to it was a Tool Comparisons gap: Playwright vs. Selenium (pillar) and The 12 best web scraping tools for 2026, priority 34,402, with both directions missing.

Priority scores in the tens of thousands look dramatic compared to the blog-only run (where the top scores sat in the 30s). The numbers are bigger because they now include in-degree contributed from across all Apify properties: a post that's referenced from dozens of docs pages and Store listings has a much larger "authority weight" than the blog-only graph could see.

That's the kind of finding a manual audit can surface if the reviewer happens to look at the right combination of posts. The auditor surfaces it deterministically because it scores every potential edge by pillar_in_degree × supporting_authority and ranks them all.

Sample interaction: fixing the top two cluster gaps

As soon as the audit finished, the natural next prompt was "fix the two top cluster gaps." Claude:

Read the cluster_gaps.csv rows for the two highest-priority edges: Python pillar ↔ "9 best Python libraries" (priority 34,407) and Playwright vs. Selenium ↔ "12 best web scraping tools" (priority 34,402).
Fetched all four involved blog posts in parallel from blog.apify.com to see the actual body text, not just metadata.
For each missing edge, scanned the source post's structure for a section where the target topic comes up naturally, e.g., for the Playwright vs. Selenium link into the "best web scraping tools" post, the prompt was "find any place that mentions browser automation, JavaScript rendering, Playwright, Selenium, or headless browsers — somewhere a link to 'Playwright vs. Selenium for web scraping' would fit naturally."
Drafted the anchor text and the surrounding sentence rewrite for each insertion.

That took a couple of minutes. A human SEO analyst could do this, but they'd need an hour or two and would need to remember to start with the highest-leverage gap rather than the first post they happened to open.

Claude reading the two top cluster gaps after the audit completes, then fetching the four involved blog posts in parallel to find natural insertion points.

Where an editor would actually start

The abstract numbers (149 orphans, 996 gaps) are useful for understanding scale but not for deciding what to do on a Tuesday morning. The auditor produces a posts_to_fix.csv sorted ascending by score, so the bottom of that list is the literal work queue. The five worst-scoring posts we saw earlier share the same root cause: nothing on the blog links to them.

None of them are bad posts. The COVID-19 piece is a genuine bit of Apify history, the screenshot tutorial is evergreen, and the Nextbrowser piece is timely (and was published very recently, hence the lack of links).

For an editor, this list is more actionable than any aggregate stat. The fix is concrete (find places to link to these from related posts), the work is bounded (one row at a time), and the win is measurable (each post's score will visibly climb on the next run).

From a project to a command: How much time this saves

Manual SEO audit: 123–184 hours (3–4.5 weeks). Output = a report/summary.
Tool-assisted (Screaming Frog + Ahrefs + human synthesis): 40–80 hours (1–2 weeks). Output = a spreadsheet/analyst notes.
This auditor: 30–60 min runtime + ~30 min review. Output = a ranked work queue and drafted anchors on demand.

The Claude Skill combined with a custom Apify Actor did the audit roughly 100x faster than doing this manually. Manual estimates above are internal ballparks based on ~10–15 min per post for a thorough single-reviewer audit; treat them as ballpark, not benchmarks. The salient point: this stops being a "project" and becomes a "command." Run it monthly. Run it before and after every content sprint. Run it when an editor asks a question about a specific post.

What it costs

Compute: runs locally, no infrastructure overhead.
LLM: ~$0.50–$2 per blog-only run, depending on model. Full-scope runs are higher because of the larger crawl, but the LLM phase still only evaluates links originating from blog posts, so the LLM bill scales sub-linearly with crawl size.
Total per run: under $5 in API spend for the blog-only pass.

For comparison: the manual equivalent at a contracted SEO rate ($75–$150/hour) would be $9,000–$27,000 per audit. We can do it monthly for less than the cost of one analyst lunch.

Caveats

The auditor doesn't replace SEO strategy. It executes the mechanics of a link audit. Deciding which clusters to invest in, which topics to seed next, and how to position pillar content remains human work.
LLM verdicts are advisory and should be spot-checked before bulk application. We've done multiple manual checks, but haven't formally measured error rate.
The auditor is tuned to our sitemap structure, tag taxonomy, and excluded-cluster rules. Pointing it at a different blog requires adaptation.
The full-scope pass is best run from Claude Code in the terminal rather than Claude Desktop, because the Actor process can still outlive Desktop's background-step timeout. This is a one-time setup (install Claude Code via npm, drop the skill into ~/.claude/skills/) but it is an extra step for editors who don't already live in a terminal.
The Store-page filter assumes editorial links to the blog only live on main Actor pages, not on Actor runs / issues / builds. That has held for our taxonomy so far. If that ever changes, the filter would need to be relaxed.

What we learned building it

The biggest advantage of all this isn't the Actor or the LLM calls. The real utility of the project came from five design choices:

Score every post on one scale (0–100). Not "100 errors" or "12 warnings." A single number per post that an editor can scan and prioritize.
Output a work queue, not a report. posts_to_fix.csv is sorted; the editor starts at the top and stops when they run out of time.
Wrap the whole thing in a conversational interface. Claude reading the CSVs and answering plain-English questions is what makes it usable by people who aren't going to open a spreadsheet on a Tuesday afternoon.
Define "orphan" honestly. A blog-only graph overstates the orphan count by treating every post that isn't linked from another blog post as orphaned, even if docs and the marketing site link to it heavily. Expanding the graph to include all Apify properties gives the truthful answer. The empirical receipt: our blog-only audit reported 302 orphans (40% of the blog). The full-scope audit reported 149 (20%). Half the "orphans" were never orphaned; they were just linked from elsewhere.
Crawl what carries signal, not what exists. Apify Store has ~230,000 sitemap URLs, but only a few thousand main Actor pages carry editorial links to the blog. Filtering the sitemap to those, plus raising concurrency, collapsed the full-scope runtime from "all afternoon" to "about as long as the blog-only run." The lesson: an honest definition is only useful if you can compute it cheaply enough to actually run.

Claude and Apify brought those choices to life, solved a problem that had no favorable solution before now, and made the solution repeatable and efficient enough to make it a routine.