[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-10-11

[github-actions[bot]]

Executive Summary

A Claude Code user (no Copilot, no Copilot CLI) can fully onboard to gh-aw today — Claude has a complete, documented auth path (API key + WIF) and full core-tool parity. But Copilot remains the privileged default: it's the only engine gh aw init auto-scaffolds a custom agent/MCP wiring for, the only one with an org-billing auth shortcut, and it dominates the example corpus. Key finding: Claude workflow examples trail Copilot 61 to 119 (1.95x), and nothing in the prerequisites section warns a Claude Code user that their claude login OAuth token will be silently rejected until they hit the error mid-setup.

Severity Findings

Critical Blockers (0) · Major Obstacles (4) · Minor Confusion (3)

Critical Blockers: None — a Claude-only user can complete setup end-to-end using ANTHROPIC_API_KEY or Anthropic WIF.

Major Obstacles

1. CLAUDE_CODE_OAUTH_TOKEN trap, undisclosed upfront. A Claude Code user naturally has run claude login; gh-aw explicitly rejects that token and points to ANTHROPIC_API_KEY/WIF instead (quick-start.mdx:151, cli.md:234). This is absent from the prerequisites list (quick-start.mdx:66-75) — discovered only after setup is attempted.
2. gh aw init scaffolding is Copilot-only. Only --engine copilot auto-creates .github/agents/agentic-workflows.md and wires MCP servers by default; Claude/Codex/Gemini/Pi get neither (cli.md:107).
3. Example/smoke-test parity gap. Workflow engine counts: Copilot 119, Codex 77, Claude 61, Pi 36 (1.95x Copilot-to-Claude ratio). Smoke-test files skew harder: ~26 copilot/codex smoke workflows vs ~6 for Claude.
4. Anthropic WIF is link-only. Copilot gets two fully inlined auth paths (org billing steps + PAT creation steps, quick-start.mdx:131-142); Anthropic WIF and Google WIF each get a single link out to the auth reference with no inline steps (quick-start.mdx:149, :169).

Minor Confusion

1. Contradictory Copilot web-fetch claims: tools.md:130 says Copilot "disables its native web tools" (offline BYOK), but tools.md:131 says web-fetch "remains supported by a custom proxy-aware tool" under copilot-sdk: true — unclear which applies by default.
2. CODEX_API_KEY precedence over OPENAI_API_KEY is stated once (quick-start.mdx:161) and never repeated in the CLI reference or troubleshooting.
3. No documented guidance on rotating ANTHROPIC_API_KEY/OPENAI_API_KEY for already-deployed workflows.

Engine & Tool Matrix

Engine	Setup	Examples	Auth	Score
Copilot	Inline, 2 paths (org billing + PAT); auto-scaffolded by gh aw init (cli.md:107)	119 workflows, ~26 smoke tests	copilot-requests: write org billing, or COPILOT_GITHUB_TOKEN PAT (quick-start.mdx:131-142)	9/10
Claude	Inline API-key steps; WIF link-only (quick-start.mdx:149); no init auto-scaffold	61 workflows, ~6 smoke tests	ANTHROPIC_API_KEY or Anthropic WIF; CLAUDE_CODE_OAUTH_TOKEN explicitly rejected (cli.md:234)	6/10
Codex	Inline API-key steps; precedence rule easy to miss (quick-start.mdx:161)	77 workflows	OPENAI_API_KEY or CODEX_API_KEY (latter takes precedence)	7/10
Custom/other (Pi, Gemini, aider, agy, cursor, goose, kiro, crush, pydantic-ai, deepseek-harness)	Pi needs manual cli-proxy/gh-proxy config (quick-start.mdx:175); 13 distinct engines have conformance tests	Pi 36; everything else ≤3 each, mostly single-instance conformance workflows	Varies by provider; Pi defers to underlying provider via model prefix	5/10

Core tool surface (GitHub, Bash, Linear, Jira, Playwright, cache-memory, repo-memory, QMD, custom MCP servers) is uniformly supported across all 5 engines per tools.md — no engine-specific gating there. The real gaps are web tools and scaffolding/auth ergonomics: web-search is disabled by default for Codex/Claude (tools.md:128), effectively unavailable for Copilot (offline BYOK, tools.md:130), and absent entirely for Gemini/Pi (tools.md:132).

Auth Gaps

• Prerequisites are auth-method-blind (quick-start.mdx:66-75): no warning that a claude login OAuth token will be rejected, before the user picks an engine.
• WIF documented asymmetrically: Copilot's two paths are fully inlined with numbered steps; Anthropic/Google WIF are single-sentence links to /gh-aw/reference/auth/ (quick-start.mdx:149, :169).
• add-wizard WIF support unclear (cli.md:155): wizard prompts for ANTHROPIC_API_KEY, but whether it can configure WIF interactively (vs. always manual) isn't stated.
• No Claude-specific troubleshooting entries — cli.md's troubleshooting section doesn't cover ANTHROPIC_API_KEY not set or WIF federation failures.
• Redundant, uncentralized OAuth-token warning: the same rejection is explained three times (quick-start.mdx:151, cli.md:155, cli.md:234) instead of once with cross-links.

Recommended Actions

Priority 1

• Add an explicit "don't use claude login's OAuth token" callout to the prerequisites section (quick-start.mdx:66-75), before engine selection.
• Extend gh aw init to scaffold an equivalent custom-agent/MCP wiring file for Claude and Codex, not just Copilot (cli.md:107).

Priority 2

• Inline the Anthropic and Google WIF setup steps in quick-start.mdx (currently just links at :149/:169), matching the depth given to Copilot's two auth paths.
• Add a cross-engine auth decision table (API key vs. WIF vs. org billing) near quick-start.mdx:129-176.

Priority 3

• Resolve the Copilot web-fetch contradiction between tools.md:130 and :131.
• Repeat the CODEX_API_KEY precedence rule in the CLI auth reference, not just quick-start.mdx:161.
• Add API-key rotation guidance for already-deployed workflows.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 165.5 AIC · ⊞ 5.2K · ◷

• expires on Oct 12, 2026, 4:59 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-10-12T12:59:48.382Z.

Closed by Workflow