> For the complete documentation index, see [llms.txt](https://faction-os.gitbook.io/faction-os-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://faction-os.gitbook.io/faction-os-docs/.spec_system/archive/sessions/phase01-session06-llm-fallback-and-scan-privacy/spec.md). # Session Specification **Session ID**: `phase01-session06-llm-fallback-and-scan-privacy` **Phase**: 01 - Core Runtime Hardening **Status**: Completed **Created**: 2026-05-29 **Package**: apps/server **Package Stack**: TypeScript, Express, Vitest, Anthropic SDK optional fallback *** ## 1. Session Overview This session hardens the server LLM and scan surfaces after the route, WebSocket, archive, export, and replay privacy boundaries from Sessions 04 and 05. The current server already exposes plan, analyze, scan, idle suggestion, and memory routes with deterministic canned fallbacks when provider credentials are absent. The remaining Phase 01 gap is explicit provider-transfer control and stronger route validation before local prompts, paths, file contents, and memory findings can cross into provider or downloaded surfaces. The work stays inside `apps/server`. It does not add a new hosted provider, change the web scan UI, or implement a unified erasure workflow. It makes the existing local-first LLM behavior release-safer by default, preserves no-key determinism, bounds scan and analysis inputs, refuses unapproved scan roots, and documents how operator-controlled provider transfer is enabled. *** ## 2. Objectives 1. Preserve deterministic no-key responses for plan, analyze, scan, idle, and memory routes. 2. Require an explicit provider-transfer opt-in before Anthropic calls can receive local data. 3. Validate LLM route inputs with compact, non-echoing errors and bounded file/scan payloads. 4. Redact prompt, path, command, token, URL, memory, and scan metadata before external transfer or broad exposure. *** ## 3. Prerequisites ### Required Sessions * [x] `phase01-session04-server-routes-and-authorization-boundaries` - provides shared route validation and auth-boundary patterns. * [x] `phase01-session05-websocket-hydration-and-archive-export-privacy` - provides shared redaction helpers and export/archive privacy posture. ### Required Tools/Knowledge * Node.js 20 or newer with npm workspaces. * Vitest server test project. * `apps/server/src/lib/requestValidation.ts` shared invalid-request envelope. * `apps/server/src/lib/sessionPrivacy.ts` redaction helper behavior. * Existing LLM engines under `apps/server/src/llm/engines`. ### Environment Requirements * No `ANTHROPIC_API_KEY` is required for test or local fallback behavior. * Provider-enabled tests must not call the network; they verify the configured boundary and fallback-blocked path. * `EXAMPLES/` remains available only as ignored source-intake evidence and must stay ignored by default scans. *** ## 4. Scope ### In Scope (MVP) * Local operator can call LLM routes without provider credentials and receive stable fallback-shaped results. * Local operator can configure a provider key without accidentally transferring local code unless `FACTIONOS_ALLOW_LLM_PROVIDER_TRANSFER=true` is set. * Local operator can scan only approved roots, with `process.cwd()` approved by default and extra roots controlled through `FACTIONOS_SCAN_ROOTS`. * Local operator receives compact validation errors for malformed LLM bodies without raw prompt, file content, token, or path echo. * Local memory and scan responses redact sensitive path, prompt, command, token, URL, and transcript-like values before broad route exposure. * Server docs describe LLM fallback, provider opt-in, scan root allowlists, local memory retention, and remaining erasure limitations. ### Out of Scope (Deferred) * New hosted LLM providers - Reason: provider expansion needs separate service and privacy review. * Transcript-mining UI redesign - Reason: this session only hardens route and memory boundaries. * Unified local archive/memory erasure command - Reason: retained under security finding M-002 for release hardening. * Web settings scan UI changes - Reason: this session targets the `apps/server` package. * Hosted analytics or account-backed storage - Reason: later phases own hosted surfaces. *** ## 5. Technical Approach ### Architecture Route validation stays at the Express boundary and follows the Session 04 `invalid_request` pattern. New LLM-specific parsing helpers should accept `unknown` bodies, validate or cap each field, and return compact field errors without echoing raw request payloads. Provider-transfer control belongs in `llmClient`. `ANTHROPIC_API_KEY` should indicate that a provider could be used, but local data should only be sent when `FACTIONOS_ALLOW_LLM_PROVIDER_TRANSFER=true` is also present. When provider transfer is disabled or unavailable, engines keep their deterministic canned fallback behavior. When provider transfer is enabled, call inputs pass through the existing deterministic redaction helper before the SDK call. Scan root validation belongs before `walkCodebase`. The walker remains a small filesystem utility, while the route decides whether a requested root is inside an approved root. Approved roots are the current working directory plus colon-delimited paths from `FACTIONOS_SCAN_ROOTS`. ### Design Patterns * Boundary validation: parse all LLM request bodies before engine invocation. * Explicit provider gate: require both provider credentials and transfer opt-in. * Defensive redaction: sanitize text before provider transfer and before memory/scan route exposure. * Deterministic fallback: blocked or unavailable provider calls return canned outputs, not opaque errors. * Compact errors: never echo raw prompt, path, file content, token, or provider error bodies. ### Technology Stack * TypeScript strict mode in `apps/server`. * Express route handlers. * Vitest Node tests. * Existing optional dynamic Anthropic SDK import. * Existing Biome formatting and linting. *** ## 6. Deliverables ### Files to Create | File | Purpose | Est. Lines | | --------------------------------------------- | ------------------------------------------------------------------------ | ---------- | | `apps/server/src/lib/llmRequestValidation.ts` | Validate and cap LLM route bodies with compact errors. | \~220 | | `apps/server/src/lib/llmPrivacy.ts` | Provider-transfer state, scan root allowlist, and LLM redaction helpers. | \~180 | ### Files to Modify | File | Changes | Est. Lines | | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ---------- | | `apps/server/src/lib/llmClient.ts` | Add explicit provider-transfer opt-in, redacted call inputs, timeout fallback, and provider state exports. | \~90 | | `apps/server/src/llm/engines/onDemandAnalysis.ts` | Redact issue fields and provider-bound file text while keeping deterministic parsing. | \~50 | | `apps/server/src/llm/engines/planDecomposer.ts` | Redact provider-bound prompt/convention text without changing local plan response shape. | \~30 | | `apps/server/src/llm/engines/idleSuggestionEngine.ts` | Redact provider-bound idle context before external transfer. | \~30 | | `apps/server/src/llm/engines/transcriptMiner.ts` | Redact provider-bound transcript context and persisted finding text. | \~30 | | `apps/server/src/routes/llm.ts` | Use validation helpers, provider privacy headers, approved scan roots, redacted scan events, and redacted memory output. | \~160 | | `apps/server/tests/llm.test.ts` | Cover no-key fallback, compact validation errors, blocked provider-transfer mode, scan roots, and redacted memory/scan output. | \~220 | | `apps/server/tests/codebaseWalker.test.ts` | Extend ignore/cap assertions for historical directories and malformed file scenarios as needed. | \~60 | | `apps/server/README_server.md` | Document LLM fallback, provider opt-in, scan allowlists, and local memory retention. | \~70 | | `docs/api/README_api.md` | Update concise LLM privacy and scan behavior. | \~60 | | `docs/api/event-api-hook-contracts.md` | Update detailed LLM route hardening and retained gaps. | \~100 | | `docs/privacy-and-security.md` | Update provider-transfer, scan, memory, and retention posture. | \~100 | *** ## 7. Success Criteria ### Functional Requirements * [ ] LLM routes return deterministic useful fallback responses without provider credentials. * [ ] Provider-bound calls require `ANTHROPIC_API_KEY` plus `FACTIONOS_ALLOW_LLM_PROVIDER_TRANSFER=true`; otherwise they fall back locally. * [ ] LLM route validation rejects malformed bodies with `invalid_request` details and does not echo raw prompts, file contents, paths, tokens, or provider errors. * [ ] Codebase scan refuses filesystem root, non-approved roots, malformed limits/extensions, and default-ignored historical directories. * [ ] Scan and memory route responses avoid broad exposure of absolute roots, prompts, URLs, command previews, tokens, and transcript-like values. * [ ] Docs clearly distinguish local fallback, provider-transfer opt-in, local memory retention, and remaining erasure limitations. ### Testing Requirements * [ ] Server LLM tests cover plan, analyze, scan, idle, and memory deterministic fallback paths. * [ ] Server LLM tests cover provider-key-present but transfer-not-enabled behavior without network calls. * [ ] Server LLM tests cover malformed route bodies, scan caps, approved roots, ignored `EXAMPLES/` paths, and redacted output. * [ ] Codebase walker tests cover ignore lists, symlink avoidance, caps, and truncation. * [ ] Focused Vitest, server typecheck, Biome check, `git diff --check`, ASCII, and LF checks pass. ### Non-Functional Requirements * [ ] No hosted service, provider SDK, or network access becomes required for local workflows or tests. * [ ] External dependency calls have a timeout and fallback path. * [ ] Provider-transfer and error logs do not expose raw prompts, local paths, file contents, tokens, or URLs. * [ ] LLM scans remain bounded by file count, file size, per-file content length, extension list, and approved root. ### Quality Gates * [ ] All files ASCII-encoded. * [ ] Unix LF line endings. * [ ] Code follows project conventions. * [ ] Biome format/lint succeeds for touched files. * [ ] TypeScript typecheck succeeds for `apps/server`. *** ## 8. Implementation Notes ### Key Considerations * `ANTHROPIC_API_KEY` should no longer be enough by itself to transfer local data. * Provider transfer remains optional and operator-controlled; local fallback is the default behavior. * Existing web scan UI sends only `{ root }`; server-side allowlist errors must be clear but compact. * Redaction should preserve useful relative file names and issue counts while removing absolute roots, secrets, URLs, prompts, commands, and transcripts. * Memory persistence remains local and append-only; this session hardens route exposure, not retention deletion. ### Potential Challenges * Provider tests must not import or call a real SDK - verify blocked-transfer behavior before dynamic import. * Scan-root allowlists can break existing tests that use temp directories - test setup should set `FACTIONOS_SCAN_ROOTS` for temp roots explicitly. * Over-redaction can make scan results useless - preserve relative paths and issue metadata where safe. * Plan fallback currently may return null if parsed output does not match engine expectations - keep existing behavior but make route tests deterministic. ### Relevant Considerations * \[P00-apps/server] **Scanner ignore posture**: keep LLM scans focused on current source and ignore `EXAMPLES`, findings, and generated artifact directories by default. * \[P00-apps/server] **Optional LLM dependency**: keep no-key fallbacks deterministic and do not require the optional SDK for local tests. * \[P00] **Prompt and path privacy**: redact prompts, paths, command previews, tokens, URLs, and transcripts before provider transfer or broad response exposure. * \[P00] **Bounded local buffers**: preserve explicit file count, file size, and content truncation limits. * \[P00] **Local-first boundary**: optional API keys must not become prerequisites for core local workflows. ### Behavioral Quality Focus Checklist active: Yes Top behavioral risks for this session: * Malformed LLM route payloads cause crashes, raw payload echoes, or broad scans. * Provider key configuration silently sends local file content, prompts, paths, or transcripts. * Scan-root validation blocks legitimate local workflows without a documented allowlist path. * Memory and scan responses expose sensitive local values after Session 05 export redaction created safer expectations. *** ## 9. Testing Strategy ### Unit Tests * LLM privacy helper tests through route-facing assertions in `apps/server/tests/llm.test.ts`. * Codebase walker ignore, caps, truncation, and symlink behavior in `apps/server/tests/codebaseWalker.test.ts`. ### Integration Tests * Real Express server tests for `/llm/plan`, `/llm/analyze`, `/llm/scan-codebase`, `/llm/idle-suggestion`, `/memory`, and `/memory/:type`. * Tests run with `ANTHROPIC_API_KEY` absent for fallback mode and present without transfer opt-in for blocked-provider mode. * Scan tests use temp directories explicitly approved through `FACTIONOS_SCAN_ROOTS`. ### Manual Testing * Start the local server with no Anthropic key and verify plan/analyze/scan/idle/memory responses are deterministic. * Set `ANTHROPIC_API_KEY` without `FACTIONOS_ALLOW_LLM_PROVIDER_TRANSFER` and verify responses stay local-fallback with provider-blocked headers. * Set `FACTIONOS_SCAN_ROOTS` to a local repo path and confirm `/llm/scan-codebase` accepts only roots under approved paths. ### Edge Cases * Missing body, array body, empty prompt, oversized prompt, malformed file list, no valid analysis files, invalid scan limit, invalid extensions, root `/`, nonexistent root, non-approved root, `EXAMPLES/` directory, symlink loop, secret-like file contents, URL in memory finding, and absolute path in analysis issue. *** ## 10. Dependencies ### External Libraries * None added. ### Other Sessions * **Depends on**: `phase01-session04-server-routes-and-authorization-boundaries`, `phase01-session05-websocket-hydration-and-archive-export-privacy` * **Depended by**: `phase01-session07-runtime-contract-documentation` *** ## Next Steps Run the implement workflow step to begin AI-led implementation. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://faction-os.gitbook.io/faction-os-docs/.spec_system/archive/sessions/phase01-session06-llm-fallback-and-scan-privacy/spec.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.