How it works

The design principle underneath every decision below is precision over recall. A false positive costs more than ten missed real breaks, because the first false positive is the last time the report gets read without suspicion. So at every stage, when a rule is ambiguous about whether something is broken, the answer is to say nothing — to skip the snippet and count it, rather than guess and risk being wrong in front of a stranger.

That trade-off explains why this page exists: it's not enough to say the tool is careful. Here is exactly what it checks, what it deliberately doesn't, and a real run's worth of numbers to show the ratio.

1. Extraction

Given a docs source — a local file, a glob of files, or a URL (including an llms-full.txt export, which many docs sites now publish as a single-request dump of their whole site) — snippetcheck parses the markdown for fenced code blocks, tracking each one's language tag, its line number in the original document, and the heading path above it (so a finding can be reported as Tool Calling > MCP Tools > SSE Transport, not just a bare line number).

2. The target-import filter

Every extracted snippet is checked for whether it actually imports the package being audited. A snippet that doesn't is skipped as no-target-import — this is usually the single largest skip category on a real docs site, because most pages on a docs site aren't about the package's own API at all (setup guides, unrelated framework examples, prose-only pages).

3. A real install

snippetcheck installs the exact package and version being audited into a disposable temp workspace with npm install — the same declarations a real consumer would get from npm install today, not a cached copy, not a guess from training data. Every snippet that survives the import filter is then written to a real file and type-checked against that real install with the TypeScript compiler API.

4. The nine-code allowlist

TypeScript can raise hundreds of distinct diagnostic codes against a documentation snippet, most of which say nothing about whether the docs are stale — an untyped placeholder variable, a missing dev dependency, a strict-mode complaint the snippet was never written to satisfy. snippetcheck discards all of them except nine, each precise enough that its message shape reliably indicates a real export, property, option, or argument count that no longer matches what the package publishes. See the error codes for all nine, each with a real example.

5. The origin gate

Passing the allowlist isn't enough on its own — a diagnostic also has to be traceable back to a symbol actually declared inside the target package's own installed files, not some unrelated type the snippet happens to reference. For a missing export or import-form mismatch, that means the module specifier in the diagnostic message has to resolve to the target package. For a missing property, unknown option, or wrong argument count, that means walking from the diagnostic's position back to the actual declaration TypeScript resolved it against, and confirming that declaration's file path sits inside node_modules/<package>/. If a diagnostic can't be traced this way, it's dropped — silently, the same as if it never happened. Dropping a real finding costs nothing. Reporting one that turns out to be about some unrelated dependency costs the reader's trust in every other line of the report.

6. Historical and before/after skips

Documentation sites keep migration guides, changelogs, and before/after comparisons — pages that intentionally show removed or outdated APIs, because showing the old way is the entire point of a migration guide. Flagging that content as broken would be technically correct and substantively wrong: it's not a stale doc, it's a doc doing its job. snippetcheck skips two kinds of content by default: snippets nested under a heading matching migration, upgrade, changelog, or a version pattern like "v4 to v5", and snippets whose fence or nearby prose says before/old/deprecated or leads with a ❌ marker. Both are opt-outable with flags, for someone auditing their own migration guide for accuracy rather than checking whether current docs are current.

7. The path sanitizer

Every diagnostic message that survives the steps above is swept for the CLI's own temp-workspace machinery before it's ever rendered: a resolved path like /private/var/folders/.../snippetcheck-x9f2/node_modules/pkg/index is collapsed back to just pkg, matching what the snippet actually wrote. If a path-shaped string survives sanitizing anyway, the finding is dropped rather than shown — the same reasoning as the origin gate: a report that leaks a stranger's own machine layout into a message sent to another stranger looks like a script someone ran once, which is the opposite of the impression a report exists to create.

A real worked example

This is the literal output of snippetcheck check https://ai-sdk.dev/llms-full.txt --package ai, against ai@7.0.66, published in full in the CLI's own README:

19 findings out of 272 checked, 4,460 found. That ratio is the argument for this whole methodology, and it does more work than any claim about rigor could: the overwhelming majority of what gets extracted from a real docs site is either not about the package at all, deliberately historical, or genuinely fine — and the tool says so, out loud, rather than going quiet about everything it didn't check.

See the nine error codes for what each of those 19 findings actually looked like, or the homepage for how to run this against your own docs.