Skiping llms.txt generation during linkcheck builds could be realized?

[Yusaku01]

Background

CI/CD wait times hinder development velocity. Reducing build latency is crucial, especially when frequent revisions are needed, to minimize context switching and merge conflict risks.

linkcheck and Netlify preview deployments are currently our biggest bottlenecks. I’m proposing a way to trim the linkcheck pipeline.

Problem

lint:linkcheck runs a full Astro build, including starlightPluginLlmsTxt(). Generating llms.txt files for every page adds significant overhead, yet these files aren't even used during link verification.

Points to consider

1. Do we actually need llms.txt files for the linkcheck process?
2. How should we handle documentation files (like src/content/docs/en/guides/build-with-ai.mdx) that link directly to llms.txt?

Regarding point 1

If llms.txt is required for link checking, this discussion isn't necessary, and I’ll close it.

Regarding point 2

If we stop generating llms.txt, existing direct links will trigger broken link errors. We’ll need a workaround for the linkcheck process, such as generating a dummy, empty llms.txt on the fly. Open to any other suggestions.

Actual experimentation

Solution at this time

• Introduce a SKIP_LLMS env var in astro.config.ts (following our existing SKIP_OG pattern).
• Create dummy llms.txt files in dist/ post-build to avoid false-positive broken link reports.

I've updated the following three files:

• astro.config.ts (updated)
• package.json (updated)
• scripts/write-linkcheck-llms-stubs.mjs (added)

In astro.config.ts, I added a conditional to enable/disable the Starlight llms.txt plugin via an environment variable.

const starlightPlugins = [
	starlightPluginSmokeTest(),
	...(process.env.SKIP_LLMS ? [] : [starlightPluginLlmsTxt()]),
];

For package.json, I've set SKIP_LLMS=true for linting. I added a build:linkcheck script, but I might just inline this into lint:linkcheck later.

"build:linkcheck": "SKIP_OG=true SKIP_LLMS=true pnpm build && node ./scripts/write-linkcheck-llms-stubs.mjs",
...
"lint:linkcheck": "pnpm build:linkcheck && node --experimental-transform-types ./scripts/lint-linkcheck.ts",

The new scripts/write-linkcheck-llms-stubs.mjs handles generating empty llms.txt files. I’m considering moving this logic into scripts/lib/linkcheck/steps/ as a proper .ts file down the road.

import { mkdir, writeFile } from 'node:fs/promises';

const distDir = new URL('../dist/', import.meta.url);
const llmsFiles = [
	['llms.txt', '# Linkcheck-only stub\n'],
	['llms-full.txt', '# Linkcheck-only stub\n'],
	['llms-small.txt', '# Linkcheck-only stub\n'],
];

await mkdir(distDir, { recursive: true });

await Promise.all(
	llmsFiles.map(([name, contents]) => writeFile(new URL(name, distDir), contents, 'utf8'))
);

result

I conducted a verification in my own repository, which is a fork of withastro/docs, to see how much time would be saved by including or excluding llms.txt.

To summarize the results, excluding generating llms.txt resulted in a time reduction of approximately 20 seconds.

• Example PR for adding llms.txt generation
• Example PR for removing llms.txt generation

Related to this theme

• perf: exclude llms plugin during smoke tests #12625 — Skip LLM plugin during smoke tests
• Consider removing llms.txt files #13006 — llms.txt maintenance discussion
• ci: make linkcheck 2 times faster #1773 — Precedent: using SKIP_OG for faster linkcheck

[delucis]

Thanks for the discussion @Yusaku01! I think for now let’s see if we merge #13538 as then no special workarounds will be needed for the link checker.

If we decide to keep llms.txt, we can consider adding a workaround like you suggest!

[Yusaku01]

@delucis

I didn’t realize there was even a PR open for 'llms.txt'! I’ll keep an eye on how it goes.
Thanks for looking into this!😌