> For the complete documentation index, see [llms.txt](https://faction-os.gitbook.io/faction-os-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://faction-os.gitbook.io/faction-os-docs/.spec_system/archive/sessions/phase11-session02-test-fixture-hardening/spec.md). # Session Specification **Session ID**: `phase11-session02-test-fixture-hardening` **Phase**: 11 - Codex Hardening and Validation **Status**: Complete **Created**: 2026-05-31 **Package**: Cross-cutting **Package Stack**: TypeScript protocol/server/web tests, JavaScript hooks/CLI tests, Vitest 4.1.7, Node 20+, Biome 2.4.16 *** ## 1. Session Overview This session hardens the Codex integration with source-backed fixtures and focused automated coverage across the packages that own the integration boundary. Phase 10 wired Codex through protocol contracts, hook maps, CLI install/uninstall paths, server ingest, and web display. Phase 11 Session 01 synchronized the docs and privacy contract. Session 02 now turns those claims into repeatable tests using synthetic, redacted Codex-shaped payloads. The work is cross-package by design. Protocol tests keep the shared vocabulary stable, hook tests verify Codex payload normalization and managed hook maps, CLI tests verify provider routing and diagnostics, server tests replay a full Codex lifecycle through `/event`, and web tests verify Codex display, filtering, replay, and export privacy behavior. Fixtures must be realistic enough to catch contract drift but sanitized enough that raw prompts, patch bodies, transcript paths, terminal output, MCP arguments, token-like values, and broad local paths never become tracked evidence. This session may fix only defects required for reliable automated coverage. It does not add hosted behavior, telemetry, plugin packaging, live provider transfer, broad UI redesign, or a manual end-to-end validation record. The next session owns live or fixture-replayed cockpit validation after this automated safety net is in place. *** ## 2. Objectives 1. Add or extend redacted Codex fixtures for lifecycle, Bash, `apply_patch`, permission requests, subagents, MCP tool names, unknown events, and compaction events where used. 2. Strengthen focused protocol, hooks, CLI, server, and web tests so Claude compatibility and Codex behavior are covered together. 3. Assert redaction and bounded-payload behavior for prompts, patch bodies, transcripts, terminal output, MCP arguments, paths, tokens, and local state. 4. Document fixture-backed limitations and keep fixes limited to blockers discovered by the new automated coverage. *** ## 3. Prerequisites ### Required Sessions * [x] `phase11-session01-documentation-and-privacy-sync` - provides the stable docs and privacy wording that fixture tests should validate against. * [x] `phase10-session01-cli-provider-selection-and-installer-split` - provides `--cli claude|codex|all` routing. * [x] `phase10-session02-codex-hook-install-uninstall-and-diagnostics` - provides Codex hook install, uninstall, status, and doctor behavior. * [x] `phase10-session03-server-ingest-codex-lifecycle` - provides Codex ingest aliases and server lifecycle handling. * [x] `phase10-session04-web-codex-awareness` - provides Codex web labels, search/filter behavior, replay, and export privacy coverage. ### Required Tools/Knowledge * npm workspace layout and package-specific test commands. * Existing Codex files under `packages/protocol`, `apps/hooks`, `apps/cli`, `apps/server`, and `apps/web`. * Current `hooks.codex.json`, hook handler normalization helpers, CLI Codex installer, server event ingest route, and web Codex fixtures. * Phase 11 PRD test matrix and Session 02 stub. * Vitest focused runs and repo quality gates from `.spec_system/CONVENTIONS.md`. ### Environment Requirements * Node 20+ and installed npm workspace dependencies. * No OpenAI, Anthropic, Codex, Claude, Cloudflare, hosted, database, or provider credentials are required. * Tests must use synthetic fixtures only. * New or changed files must be ASCII-only with Unix LF line endings. *** ## 4. Scope ### In Scope (MVP) * Maintainers can rely on shared protocol tests for Codex hook event names and `codex-cli` hero vocabulary - verify shared constants, type aliases, and backward-compatible Claude aliases. * Hook runtime maintainers can replay Codex-shaped payloads safely - add fixture-backed tests for lifecycle, prompt, Bash, `apply_patch`, permission, subagent, MCP, spool, and log sanitization. * CLI maintainers can trust provider routing coverage - harden tests for `init --cli claude`, `init --cli codex`, `init --cli all`, malformed hooks, idempotent reinstall, uninstall, status, and doctor output. * Server maintainers can replay a full Codex lifecycle - extend fixture coverage for Bash, `apply_patch`, permission, subagent, unknown named events, MCP-style tool names, compaction events where routed, and `cli: "codex-cli"` persistence. * Web maintainers can verify Codex-shaped data safely - cover hero cards, hero detail, mission filters, tool categorization, replay redaction, and export redaction with sanitized Codex fixtures. * Security reviewers can inspect fixture limitations - record limitations in implementation notes when a live Codex shape is approximated by a source-backed synthetic fixture. ### Out of Scope (Deferred) * Live Codex manual validation - *Reason: Phase 11 Session 03 owns end-to-end validation.* * New Codex features beyond the integration definition of done - *Reason: this session hardens existing behavior.* * Plugin packaging or project-local trusted `.codex/` setup - *Reason: Phase 12 owns packaging decisions.* * Hosted auth, hosted storage, analytics, push, remote access, public replay hosting, production-hosted validation, or external telemetry - *Reason: those remain disabled, unavailable, deferred, or no-claim.* * Broad refactors unrelated to fixture-backed test reliability - *Reason: this session should fix only blockers discovered by the focused tests.* *** ## 5. Technical Approach ### Architecture Use a fixture-first approach. Codex payload builders should live beside the tests that consume them and should be synthetic, bounded, and redacted by construction. Protocol remains the shared contract owner for `HeroCli`, hook event names, and ingest aliases. Hooks normalize provider payloads before sending local events or writing spool entries. CLI tests exercise provider routing through isolated temporary homes. Server tests replay the fixture sequence through the real Express event route. Web tests consume sanitized hero, mission, replay, and export fixtures through existing component and helper tests. The test plan should preserve Claude behavior while adding Codex confidence. Any failing assertions should first be evaluated as either a real integration blocker, an over-broad fixture, or a stale test expectation. Source fixes are allowed only where a fixture exposes a blocker in the existing Codex integration contract. ### Design Patterns * Protocol-first assertions: shared vocabulary and alias compatibility are tested before package-specific behavior. * Synthetic redacted fixtures: realistic keys and event order without raw prompt, patch, transcript, output, token, or path material. * Boundary-specific redaction tests: hooks, server, replay, export, status, and doctor each assert their own sensitive-field handling. * Isolated homes: CLI and hook tests use temp directories and never read or write real user config. * Focused runs before broad gates: run package-targeted Vitest suites first, then workspace quality checks. ### Technology Stack * Node 20+ npm workspaces. * TypeScript packages: `packages/protocol`, `apps/server`, `apps/web`. * JavaScript packages: `apps/hooks`, `apps/cli`. * Vitest 4.1.7 with node and happy-dom projects. * Biome 2.4.16 for formatting and lint checks. *** ## 6. Deliverables ### Files to Create | File | Purpose | Est. Lines | | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------ | ---------- | | `apps/hooks/tests/codexPayloadFixtures.js` | Shared synthetic Codex hook payload builders for lifecycle, tools, permissions, subagents, MCP, unknown, and compaction cases. | \~220 | ### Files to Modify | File | Changes | Est. Lines | | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ---------- | | `packages/protocol/tests/heroes.test.ts` | Assert `codex-cli` remains in shared hero CLI vocabulary and display metadata. | \~30 | | `packages/protocol/tests/ingest.test.ts` | Assert Codex hook events, compact aliases, and Claude compatibility aliases. | \~70 | | `apps/hooks/tests/codexHookMap.test.js` | Harden Codex hook map assertions for events, commands, env injection, Windows commands, timeouts, and handlers. | \~60 | | `apps/hooks/tests/hookRuntime.test.js` | Replay shared Codex fixtures through handlers and assert sanitized spool payloads. | \~140 | | `apps/hooks/tests/hookNormalization.test.js` | Extend normalization and spool sanitization coverage for Codex-sensitive fields and compaction-like payloads. | \~80 | | `apps/hooks/tests/hookPayloads.test.js` | Keep static handler source assertions aligned with fixture-backed expectations. | \~50 | | `apps/cli/tests/cliRuntime.test.js` | Harden provider-aware init/install/uninstall/status/doctor runtime coverage. | \~110 | | `apps/cli/tests/codexInstaller.test.js` | Extend malformed, idempotent, backup, CODEX\_HOME, and uninstall edge-case coverage. | \~80 | | `apps/server/tests/fixtures/event-ingest/codexLifecycle.ts` | Extend full lifecycle fixture with MCP, unknown, compaction-safe, and redacted metadata cases. | \~90 | | `apps/server/tests/eventIngest.test.ts` | Add fixture replay, unknown event, MCP, compaction, and leakage assertions. | \~130 | | `apps/server/tests/fixtures/event-ingest/README.md` | Note fixture source, sanitization boundaries, and limitations. | \~35 | | `apps/web/tests/codexAwarenessFixtures.ts` | Extend web Codex fixtures for display, tool, replay, and export privacy tests. | \~70 | | `apps/web/tests/HeroCard.test.tsx` | Add Codex hero card display coverage with accessible provider labels. | \~45 | | `apps/web/tests/HeroDetailDrawer.test.tsx` | Add Codex detail display and external model coverage. | \~45 | | `apps/web/tests/MissionLogFilter.test.tsx` | Add Codex mission filtering and tool-name coverage. | \~60 | | `apps/web/tests/toolUsage.test.ts` | Assert `apply_patch`, Bash, and MCP tool categorization. | \~40 | | `apps/web/tests/replayLink.test.ts` | Assert Codex-shaped replay private fields are redacted. | \~55 | | `apps/web/tests/exportSession.test.ts` | Assert Codex-shaped export errors redact private fields. | \~45 | *** ## 7. Success Criteria ### Functional Requirements * [ ] Protocol tests confirm `HeroCli` includes `codex-cli`, Codex lifecycle events are in `AGENT_HOOK_EVENT_NAMES`, and deprecated Claude aliases still compile. * [ ] Hook tests confirm `hooks.codex.json` maps Codex events to expected managed commands with `FACTIONOS_CLI=codex-cli` and resolved package roots. * [ ] Hook runtime tests replay `SessionStart`, `UserPromptSubmit`, Bash, `apply_patch`, `PermissionRequest`, `SubagentStart`, `SubagentStop`, MCP, unknown, and compaction-safe payloads without leaking raw sensitive fields. * [ ] CLI tests cover Claude, Codex, and all-provider install flows, malformed Codex hooks, idempotent reinstall, uninstall, status, and doctor output. * [ ] Server tests replay a full Codex lifecycle and verify hero persistence, mission lifecycle, Bash, `apply_patch`, permission, subagent, MCP, unknown event, and bounded fallback behavior. * [ ] Web tests cover Codex hero labels, hero detail, mission filters, tool categorization, replay redaction, and export redaction. * [ ] Any required source fixes are limited to defects exposed by the fixture-backed tests. ### Testing Requirements * [ ] Focused protocol, hooks, CLI, server, and web Vitest suites pass. * [ ] Workspace typecheck passes for packages with typecheck scripts. * [ ] Biome format and lint checks pass for touched files. * [ ] `git diff --check` passes. ### Non-Functional Requirements * [ ] Fixtures do not include raw prompts, patch bodies, transcript bodies, terminal output, MCP argument bodies, token-like values, provider credentials, or broad absolute paths. * [ ] Diagnostics and test assertions use labels, counts, booleans, compact IDs, and redacted paths only. * [ ] Claude Code behavior remains backward-compatible. * [ ] Hosted identity, trusted erasure, production-hosted validation, plugin packaging, and external telemetry remain no-claim. ### Quality Gates * [ ] All files ASCII-encoded. * [ ] Unix LF line endings. * [ ] Code follows project conventions. * [ ] Secret and sensitive-string review passes for new fixture material. *** ## 8. Implementation Notes ### Key Considerations * Existing code already contains partial Codex fixtures and tests. This session should extend and consolidate them instead of creating parallel, inconsistent fixture shapes. * Hook and CLI tests must use temp homes and explicit environment overrides so they never touch real `CODEX_HOME`, `~/.codex`, `~/.claude`, or `~/.factionos` state. * `apply_patch` fixtures should assert file counts, line counts, and safe touched paths, not raw patch bodies. * MCP fixtures should assert allowlisted tool names and field counts, not raw argument bodies. * Unknown and compaction-like events should stay bounded opaque or no-op paths unless a current source-backed route already supports them. * Web fixtures should reuse protocol-backed labels and should not assert raw `codex-cli` fallback text where the user-facing label should be `Codex CLI`. ### Potential Challenges * Existing coverage may already pass some matrix items: mitigate by extending assertions to close privacy and edge-case gaps rather than duplicating identical tests. * Fixture realism can drift into sensitive examples: mitigate by using sentinel strings only inside assertions that prove redaction and by checking the final serialized output does not contain them. * Cross-package failures can look like scope creep: mitigate by fixing only blockers that prevent the fixture-backed contract from passing. * Long focused test commands can be brittle: mitigate by documenting exact command groups in implementation notes after execution. ### Relevant Considerations * \[P10-packages/protocol+apps/web] **Codex display labels need one shared source of truth**: fixture tests should assert protocol-backed labels and prevent local fallback drift. * \[P03-apps/cli+apps/hooks] **Lifecycle state is local and file-based**: CLI and hook tests should isolate local homes and keep stale-state behavior covered. * \[P07] **Redaction is boundary-specific**: each boundary should assert its own sensitive-field minimization rather than relying on a single broad regex. * \[P03-packages/protocol] **Protocol leads cross-package work**: shared hook and hero vocabulary tests should lead package-specific assertions. * \[P09-apps/hooks] **Do not forward raw Codex payloads**: patch bodies, terminal output, transcript paths, token-like values, and MCP arguments must stay summarized or redacted. * \[P10-apps/cli+apps/hooks] **Backup-first Codex hook management worked**: installer and uninstall coverage should preserve user-owned hooks and first backup behavior. * \[Security] **P06-S07-HOSTED-IDENTITY**: this session does not add hosted identity. * \[Security] **P06-S07-ERASURE**: this session does not claim trusted unified erasure. * \[Security] **P06-S07-HOSTED-VALIDATION**: this session does not create production-hosted validation claims. ### Behavioral Quality Focus Checklist active: Yes Top behavioral risks for this session: * Fixture tests accidentally preserve sensitive Codex payload material in tracked files or failure snapshots. * Provider-specific tests pass while Claude compatibility aliases or default Claude setup regress. * Cross-package fixture shapes drift from the shared protocol contract and hide integration failures until manual validation. * CLI and hook tests accidentally touch real user-level configuration instead of isolated temp homes. *** ## 9. Testing Strategy ### Unit Tests * Protocol: `packages/protocol/tests/heroes.test.ts`, `packages/protocol/tests/ingest.test.ts`. * Hooks: `apps/hooks/tests/codexHookMap.test.js`, `apps/hooks/tests/hookRuntime.test.js`, `apps/hooks/tests/hookNormalization.test.js`, `apps/hooks/tests/hookPayloads.test.js`. * CLI: `apps/cli/tests/cliProvider.test.js`, `apps/cli/tests/codexInstaller.test.js`, `apps/cli/tests/cliRuntime.test.js`. * Web helpers and components: `apps/web/tests/heroCli.test.ts`, `apps/web/tests/HeroCard.test.tsx`, `apps/web/tests/HeroDetailDrawer.test.tsx`, `apps/web/tests/MissionLogFilter.test.tsx`, `apps/web/tests/toolUsage.test.ts`, `apps/web/tests/replayLink.test.ts`, `apps/web/tests/exportSession.test.ts`. ### Integration Tests * Server event route replay: `apps/server/tests/eventIngest.test.ts` using `apps/server/tests/fixtures/event-ingest/codexLifecycle.ts`. * Cross-boundary fixture replay from hooks to server-shaped payload assertions where practical. ### Manual Testing * No live manual Codex validation is required for this session. * Review fixture files and serialized test outputs for sensitive sentinel leakage. * Record any source-backed limitations in `implementation-notes.md` during implementation. ### Edge Cases * `init --cli claude` must not create Codex hook state. * `init --cli codex` must not mutate Claude settings. * `init --cli all` must be idempotent and must not duplicate managed hooks. * Malformed Codex `hooks.json` must be backed up or left untouched according to install/uninstall behavior. * `status` and `doctor` must not print raw hook commands, prompts, tokens, broad paths, or hook payloads. * Missing or malformed `tool_input`, `tool_response`, transcript, agent, and MCP fields must degrade compactly. * Unknown Codex event names must not crash server ingest or leak raw payload bodies. * `apply_patch` must report bounded edit metadata without patch bodies. * Web replay and export errors must redact Codex-shaped private fields in JSON and text envelopes. *** ## 10. Dependencies ### External Libraries * Vitest: `^4.1.7` * React testing utilities already present in `apps/web` tests. * No new runtime dependencies are planned. ### Other Sessions * **Depends on**: `phase11-session01-documentation-and-privacy-sync`, `phase10-session01-cli-provider-selection-and-installer-split`, `phase10-session02-codex-hook-install-uninstall-and-diagnostics`, `phase10-session03-server-ingest-codex-lifecycle`, `phase10-session04-web-codex-awareness` * **Depended by**: `phase11-session03-end-to-end-validation-and-integration-fixes`, `phase12-session01-codex-plugin-packaging`, `phase12-session02-release-closeout-and-rollout-readiness` *** ## Next Steps Run the implement workflow step to begin AI-led implementation. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://faction-os.gitbook.io/faction-os-docs/.spec_system/archive/sessions/phase11-session02-test-fixture-hardening/spec.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.