Skip to content

feat(llms): list all docs pages in llms.txt and add a coverage check - #239

Open
thzgajendra wants to merge 2 commits into
gofr-dev:mainfrom
thzgajendra:feat/llms-txt-full-coverage
Open

thzgajendra wants to merge 2 commits into
gofr-dev:mainfrom
thzgajendra:feat/llms-txt-full-coverage

Conversation

@thzgajendra

@thzgajendra thzgajendra commented Oct 7, 2026 •

Copy link
Copy Markdown

Description:

What changed in llms.txt:

  • New AI and agents section at the top: MCP, LLM and streaming guides, each with a one-line description.
  • New Production guides section with the 18 /docs/guides/* pages, each with a one-line description.
  • Added to existing sections: 13 advanced-guide pages, Cloud SQL, Elasticsearch migrations, the four gofr CLI subcommand pages, the /comparison and /migrate indexes, the /docs home, and llms-full.txt.
  • The MCP entry describes it as read-only (GET/HEAD/OPTIONS/QUERY) per the docs; it does not repeat the page's "validation" wording (see MCP docs page may overstate validation; verify and correct #236).

Not included: docs/quick-start/cli/page.md in gofr is an empty file, so it is not listed. It looks like a stray file and may be worth a separate look.

Breaking Changes (if applicable):

  • None. Content-only change to a static file.

Additional Information:

  • Coverage checked by comparing every non-empty page.md route in gofr-dev/gofr@development docs/ against the links in llms.txt: before, 39 of 105 pages were missing; after, all 105 are listed.
  • All 113 unique URLs in the new llms.txt return HTTP 200 on https://gofr.dev (checked with curl).
  • The issue counted 30 missing pages; the comparison found 9 more (injecting-databases-drivers, datasources/migrations/elasticsearch, four gofrcli pages, /comparison, /migrate, /docs) and these are included too.
  • No dependencies added.
  • Related: Add AI features (MCP, LLM) to AGENTS.md and README #232 (AGENTS.md/README), which lives in gofr-dev/gofr.

llms.txt linked 66 of 105 docs pages and omitted the MCP and LLM guides and all production guides. List every page, add an AI and agents section and a production guides section, and add utils/check-llms-coverage.mjs to fail when a docs page is missing.
Keep this change to llms.txt content only.

@Umang01-hash Umang01-hash left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at head 9fa1c0f. Verified all 40 newly-added doc URLs resolve (200) — no dead links, which is the key safety property for llms.txt. No PR-introduced duplicates (the two AGENTS.md occurrences are a pre-existing prose mention + list entry, not added here).

Nit: the title says "add a coverage check", but the PR is public/llms.txt only — there's no check. Since the docs content isn't in this repo, completeness ("list all docs pages") can't be enforced without that guard, so either add the check or adjust the title. Content LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

llms.txt is missing 30 of 76 docs pages, including the MCP and LLM guides

2 participants