> For the complete documentation index, see [llms.txt](https://faction-os.gitbook.io/faction-os-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://faction-os.gitbook.io/faction-os-docs/.spec_system/archive/sessions/phase18-session08-on-demand-analysis-and-project-scan-engines/spec.md).

# Session Specification

**Session ID**: `phase18-session08-on-demand-analysis-and-project-scan-engines` **Phase**: 18 - Quest Board Suggestion Parity **Status**: Complete **Created**: 2026-06-10 **Package**: apps/server **Package Stack**: TypeScript

***

## 1. Session Overview

This session adds the remaining server-side Quest Board engines after the manager, summaries, codebase scanners, and scan orchestration are in place. It implements on-demand analysis and project-level strategic scan workflows that produce canonical `AnalysisResult` and `ProjectScan` entries through `SuggestionManager`, so later web sessions can render typed cards without special server state.

The work stays local-first and server-owned. Analysis selects a bounded file set from recent git history, applies the existing LLM privacy root boundary, filters malformed or noisy model output, and caches fresh results for 30 minutes. Project scan generates 2-4 strategic items with the 24-hour freshness window used by the protocol. Both engines expose trigger and status routes under root and `/api`, use single-flight guards, and emit canonical `suggestion_update` snapshots after manager mutations.

The existing `/llm/analyze` route remains a lower-level caller-provided LLM utility unless implementation proves migration is safer. The new `/suggestions/analyze` and `/suggestions/project-scan` routes are the Quest Board persistence path and are removed from the unsupported-route classifier only after they ship.

***

## 2. Objectives

1. Implement on-demand analysis file selection, parsing, validation, freshness, and persistence through `SuggestionManager`.
2. Implement project scan generation, parsing, validation, freshness, startup staleness behavior, and persistence through `SuggestionManager`.
3. Ship root and `/api` trigger/status routes for analysis and project scan with validation, rate-limit inheritance, single-flight guards, and compact errors.
4. Add focused server tests and docs proving local-first privacy, fallback, freshness, status, and unsupported-route reconciliation.

***

## 3. Prerequisites

### Required Sessions

* [x] `phase18-session02-server-suggestion-manager-and-persistence` - provides manager-owned persisted Quest Board state and snapshot emission.
* [x] `phase18-session05-session-summary-engine-and-noise-filtering` - provides analysis noise filtering and summary lifecycle patterns.
* [x] `phase18-session07-scan-orchestration-and-quest-ignore-patterns` - provides scan status conventions, startup stale-scan wiring, route patterns, and unsupported-route reconciliation.

### Required Tools/Knowledge

* Node 26.2.0 or newer with npm workspaces.
* Existing `@factionos/protocol` suggestion analysis, project scan, freshness, and scan status contracts.
* Existing `SuggestionManager`, `analysisNoiseFilter`, `llmClient`, `llmPrivacy`, `unsupportedRoutes`, and suggestion route patterns.
* Phase 18 PRD Appendix A as traceability evidence only; no historical code, bundle, or asset import.

### Environment Requirements

* Work from the repository root.
* Use package-relative monorepo paths from the repo root.
* Keep all generated and edited files ASCII-only with Unix LF line endings.
* Provider transfer remains disabled unless both `ANTHROPIC_API_KEY` and `FACTIONOS_ALLOW_LLM_PROVIDER_TRANSFER=true` are set.

***

## 4. Scope

### In Scope (MVP)

* Operators can trigger on-demand analysis through `POST /suggestions/analyze` and read state through `GET /suggestions/analyze/status`.
* Operators can trigger project scan through `POST /suggestions/project-scan` and read state through `GET /suggestions/project-scan/status`.
* Analysis file selection uses recent git history over the last commits with a working-tree fallback, code-extension filtering, dependency and lockfile exclusion, deterministic ordering, and a cap of 10 files.
* Analysis parsing validates item type, priority, title, description, prompt, optional file path, and optional line using protocol shapes.
* Analysis output applies the Session 05 noise filter before storage and uses the 30-minute freshness window.
* Project scan output validates 2-4 strategic items with title, description, prompt, category, effort, and priority, capped by protocol limits.
* Project scan uses the 24-hour freshness window and schedules a non-blocking stale startup run alongside codebase scan startup wiring.
* Both engines update manager-owned status, prevent duplicate triggers while in flight, return fresh cached results without re-running, and map failures to compact status/errors.
* LLM calls go through `llmClient`, respect `llmPrivacy`, sanitize prompts and paths, have bounded timeouts, and fall back to schema-valid local results.
* Shipped analyze and project-scan routes are removed from `unsupportedRoutes.ts` while unrelated planned routes stay classified.
* Focused tests cover file selection, parsing, validation, freshness, status, route behavior, startup scheduling, fallback, and redaction boundaries.

### Out of Scope (Deferred)

* Quest Board UI trigger buttons and status controls - *Reason: Session 10 owns Quest actions and keyboard shortcuts.*
* Typed multi-source Quest Board card rendering - *Reason: Session 09 owns web card normalization and rendering.*
* Historical entitlement gating - *Reason: Phase 18 explicitly excludes the historical suggestion-engine feature gate.*
* Historical suggestion telemetry - *Reason: analytics capture remains disabled by default and excluded for this phase.*
* Accepting codebase issues as quests - *Reason: Phase 18 parity keeps codebase issues dismiss-only.*

***

## 5. Technical Approach

### Architecture

Add an analysis orchestration boundary in `apps/server/src/lib/onDemandAnalysisEngine.ts`. It will select files using bounded git helpers, build a sanitized prompt, call `llmClient`, parse and validate protocol-shaped `AnalysisItem` entries, apply `filterAnalysisResult`, store the result through `SuggestionManager.setAnalysisResult`, update manager-owned scan status with scope `analysis`, and emit `suggestion_update`.

Add a project scan boundary in `apps/server/src/lib/projectScanEngine.ts`. It will build a sanitized project summary prompt from safe local repository signals, call `llmClient`, parse protocol-shaped `ProjectScanItem` entries, fall back to deterministic strategic items, store through `SuggestionManager.setProjectScan`, update manager-owned scan status with scope `project`, and schedule non-blocking stale startup work.

Extend `apps/server/src/routes/suggestions.ts` with the four suggestion-engine routes. The existing `/llm/analyze` route remains available as a low-level LLM utility; the new routes own Quest Board persistence and status. Extend `apps/server/src/server.ts` to instantiate and destroy the engines, mirroring the Session 07 codebase scan lifecycle.

### Design Patterns

* Manager-owned state: all stored analysis and project scan results go through `SuggestionManager`.
* Single-flight lifecycle: duplicate triggers return current status or compact conflict instead of racing provider calls or manager mutations.
* Freshness-first execution: fresh cached results return without provider work.
* Pure parsing before side effects: git output parsing, model response parsing, validation, and noise filtering stay fixture-testable.
* Local-first privacy: no provider transfer without two-level opt-in, no raw prompts, no absolute paths, no raw command logs, and no hosted persistence.
* Explicit unsupported-route reconciliation: only de-list shipped routes.

### Technology Stack

* TypeScript in `apps/server`.
* Existing Express 5 route mounting under root and `/api`.
* Existing `@factionos/protocol` suggestion contracts.
* Existing `llmClient`, `llmPrivacy`, and `analysisNoiseFilter` helpers.
* Vitest server tests.

***

## 6. Deliverables

### Files to Create

| File                                               | Purpose                                                                                                             | Est. Lines |
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | ---------- |
| `apps/server/src/lib/onDemandAnalysisEngine.ts`    | Analysis file selection, single-flight orchestration, parsing, fallback, freshness, status, and manager persistence | \~420      |
| `apps/server/src/lib/projectScanEngine.ts`         | Project scan orchestration, parsing, fallback, freshness, startup stale scan, status, and manager persistence       | \~380      |
| `apps/server/src/llm/prompts/project-scan.md`      | Maintained auditable system prompt for project scan items                                                           | \~80       |
| `apps/server/tests/onDemandAnalysisEngine.test.ts` | File selection, parsing, validation, freshness, fallback, status, and noise-filter tests                            | \~260      |
| `apps/server/tests/projectScanEngine.test.ts`      | Project scan parsing, freshness, fallback, single-flight, startup, and status tests                                 | \~260      |

### Files to Modify

| File                                          | Changes                                                                                                    | Est. Lines |
| --------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ---------- |
| `apps/server/src/routes/suggestions.ts`       | Add analyze and project-scan trigger/status routes with validation, conflict handling, and compact errors  | \~220      |
| `apps/server/src/server.ts`                   | Instantiate analysis/project engines, schedule project startup stale scan, and destroy engines on shutdown | \~100      |
| `apps/server/src/lib/unsupportedRoutes.ts`    | Remove shipped analyze and project-scan route prefixes from planned unsupported classification             | \~30       |
| `apps/server/src/lib/llmClient.ts`            | Add project scan engine model/timeout support while preserving local fallback behavior                     | \~60       |
| `apps/server/src/llm/prompts/index.ts`        | Export the maintained project scan prompt                                                                  | \~20       |
| `apps/server/tests/suggestionRoutes.test.ts`  | Cover analyze/project route validation, cached responses, conflicts, root and `/api` prefixes              | \~220      |
| `apps/server/tests/unsupportedRoutes.test.ts` | Update planned route expectations after analyze/project routes ship                                        | \~50       |
| `apps/server/README_server.md`                | Document Quest Board analysis/project routes, freshness, startup behavior, and privacy limits              | \~90       |

***

## 7. Success Criteria

### Functional Requirements

* [ ] `POST /suggestions/analyze` validates input, starts or returns fresh analysis, prevents duplicate in-flight runs, persists filtered analysis items, and emits `suggestion_update`.
* [ ] `GET /suggestions/analyze/status` returns accurate analysis status, last-start, last-complete, and last-error fields.
* [ ] Analysis file selection is bounded to approved local roots, excludes dependency and lockfile paths, caps files at 10, and handles empty git history with deterministic fallback behavior.
* [ ] Analysis result parsing accepts only protocol-valid item types and priorities, rejects malformed or noisy self-talk items, and caps output.
* [ ] `POST /suggestions/project-scan` validates input, starts or returns fresh project scan results, prevents duplicate in-flight runs, persists project items, and emits `suggestion_update`.
* [ ] `GET /suggestions/project-scan/status` returns accurate project status, last-start, last-complete, and last-error fields.
* [ ] Project scan parses and stores 2-4 protocol-valid strategic items with category, effort, and priority.
* [ ] Startup project stale scan runs without blocking server readiness when the 24-hour freshness window has lapsed.
* [ ] Analyze and project-scan routes are no longer classified as unsupported.

### Testing Requirements

* [ ] Unit tests cover analysis file selection, parsing, noise filtering, freshness, fallback, status transitions, and single-flight behavior.
* [ ] Unit tests cover project scan parsing, freshness, fallback, status transitions, startup scheduling, and single-flight behavior.
* [ ] Route tests cover validation, root and `/api` mounting, conflicts, cached responses, status responses, and compact error redaction.
* [ ] Unsupported-route tests reflect the shipped route set.

### Non-Functional Requirements

* [ ] LLM/provider transfer still requires both provider key and explicit transfer opt-in.
* [ ] Route errors and logs do not expose raw prompts, provider payloads, absolute paths, request roots, tokens, or raw command output.
* [ ] Long-running engines use bounded timeout, cleanup on shutdown, and deterministic fallback results.

### Quality Gates

* [ ] All files ASCII-encoded.
* [ ] Unix LF line endings.
* [ ] Code follows project conventions.
* [ ] Focused server tests pass.
* [ ] `npm --workspace apps/server run typecheck` passes.

***

## 8. Implementation Notes

### Key Considerations

* Existing protocol contracts already define `AnalysisResult`, `ProjectScan`, freshness helpers, and scan status scopes; avoid inventing parallel shapes.
* `SuggestionManager.setAnalysisResult` already applies the Session 05 noise filter. The engine should still parse and validate before calling it so failure paths are compact and testable.
* The single `scanStatus` field can represent `analysis` or `project` scope for these runs. Route responses should be explicit about the current scope.
* `/llm/analyze` is lower-level and does not persist Quest Board state; keep it unless migration would reduce duplication without changing current behavior.
* The historical EXAMPLES files are evidence only. Use them for behavior and naming traceability, not code or prompt import.

### Potential Challenges

* Git history can be unavailable or shallow: fall back to working-tree changes and then deterministic local project hints.
* Provider output can be malformed or empty: parse defensively and fall back to schema-valid local items.
* Scan status is shared across codebase, analysis, and project scopes: tests should verify route responses remain correct after each engine updates scope.
* Provider timeout needs a longer budget than existing short calls: preserve default `llmClient` behavior while allowing these engines to opt into a bounded long-running timeout.

### Relevant Considerations

* \[P01-apps/server] **Anthropic transfer is two-level opt-in**: analysis and project scan prompts must not leave the process without both provider key and `FACTIONOS_ALLOW_LLM_PROVIDER_TRANSFER=true`.
* \[P03-apps/server] **Local server boundary must stay conservative**: all new routes need validation, inherited rate limits, body caps, and compact errors.
* \[P07] **Redaction is boundary-specific**: analysis/project text, file hints, persisted snapshots, logs, and route responses need explicit minimization.
* \[P03] **Stable docs are the current contract**: update server/API-adjacent docs when the shipped route set changes.
* \[P17-apps/server] **Manager-owned persistence and cleanup**: keep state, freshness, debounced persistence, and shutdown cleanup under server-owned managers and lifecycle boundaries.

### Behavioral Quality Focus

Checklist active: Yes Top behavioral risks for this session:

* Duplicate manual trigger and startup scan execution racing manager writes.
* Provider or git command failures leaving stale `running` status.
* Malformed model output or self-talk becoming persistent Quest Board cards.
* Absolute paths or raw prompt/code snippets leaking into persisted state, logs, route responses, or provider prompts.

***

## 9. Testing Strategy

### Unit Tests

* Analysis file selection with committed, staged, working-tree, empty, and invalid git outputs.
* Analysis response parsing, item caps, enum validation, noise filtering, and deterministic fallback results.
* Project scan response parsing, 2-4 item cap, category/effort/priority validation, and deterministic fallback results.
* Freshness windows, status transitions, single-flight guards, and manager persistence for both engines.

### Integration Tests

* Root and `/api` route coverage for trigger/status routes.
* Validation errors for invalid body, invalid root, invalid force flags, and compact no-echo behavior.
* Conflict responses while analysis or project scan is already running.
* Unsupported route classification after route shipping.

### Manual Testing

* Start the local server, trigger `/suggestions/analyze` and `/suggestions/project-scan`, then verify `/suggestions/summary` counts and WebSocket `suggestion_update` snapshots contain analysis/project entries.
* Re-trigger within freshness windows and verify cached results return without a new provider call.

### Edge Cases

* No git repository or git command failure.
* No selected files.
* Provider transfer disabled with provider key present.
* Provider timeout or malformed JSON.
* Fresh cached result present but stale status from a previous failed run.
* Shutdown while a project startup scan is scheduled or running.

***

## 10. Dependencies

### External Libraries

* None expected. Use existing Node, TypeScript, Express, Vitest, and `@factionos/protocol` dependencies.

### Other Sessions

* **Depends on**: `phase18-session02-server-suggestion-manager-and-persistence`, `phase18-session05-session-summary-engine-and-noise-filtering`, `phase18-session07-scan-orchestration-and-quest-ignore-patterns`
* **Depended by**: `phase18-session09-web-quest-board-typed-cards`, `phase18-session10-quest-actions-and-keyboard-shortcuts`, `phase18-session11-validation-documentation-and-handoff`

***

## Next Steps

Run the implement workflow step to begin AI-led implementation.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://faction-os.gitbook.io/faction-os-docs/.spec_system/archive/sessions/phase18-session08-on-demand-analysis-and-project-scan-engines/spec.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.