diff --git a/CONTEXT.md b/CONTEXT.md index 02ce111..4226acd 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -8,11 +8,16 @@ extensions that inspect and re-evaluate Bash commands before they are allowed. ### Authority **Enforce authority**: -Allow-only delegation — when the Judge's verdict is `allow` and every gate in -the Enforce truth table passes, the command runs without the human dialog. The -Judge can never answer `deny` with authority; every uncertain case (defer, -deny, preflight, infrastructure failure) falls back to the human dialog. -_Avoid_: AI takeover, auto-deny, full delegation +Allow-only delegation under a user-assumed-risk contract (ADR 0008): +writing `mode: "enforce"` in config v2 consents to the selected judge model +approving ordinary operations on the user's behalf. When the Judge's verdict +is `allow` and every fail-closed health gate in the Enforce truth table +passes (audit, telemetry, result kind, review acknowledgement, +generation-current), the command runs without the human dialog. The Judge can +never answer `deny` with authority; every uncertain case (defer, deny, +preflight, infrastructure failure) falls back to the human dialog. +_Avoid_: model safety certification (dropped — see ADR 0008), AI takeover, +auto-deny, full delegation **Audit log (Judge-owned)**: The Enforce-era accountability record written by the Judge package itself — @@ -21,6 +26,16 @@ record, self-checked for health; an unhealthy audit log refuses authority. _Avoid_: host contract (dropped — see ADR 0006), acknowledged write (upstream sense) +**High-risk override**: +A narrow code-level preflight recognizing clear-cut command shapes in four +categories — data loss/history rewrite, publish/deploy/infrastructure +destruction, privilege escalation/system modification, direct credential +access — that always defers to the human dialog regardless of verdict. In +Enforce it skips the model call entirely; in Shadow the model is still called +for quality observation. Deliberately not a sandbox: only well-known explicit +shapes, no alias/script/variable analysis, no `alwaysPrompt` config. +_Avoid_: command sandbox, alwaysPrompt, opaque-command blanket defer + **Irreversibility boundary**: An allow verdict requires every operation's effects to be recoverable — reversible, or reproducible from the repository or the evidence at hand. diff --git a/docs/adr/0008-enforce-user-assumed-risk.md b/docs/adr/0008-enforce-user-assumed-risk.md new file mode 100644 index 0000000..90e7152 --- /dev/null +++ b/docs/adr/0008-enforce-user-assumed-risk.md @@ -0,0 +1,98 @@ +--- +status: accepted +--- + +# Enforce as user-assumed risk — promotion gates leave the runtime + +PIEXTENSIO-23. This ADR supersedes the **runtime effect** of the +per-segment promotion floor (PIEXTENSIO-10, as consumed by PIEXTENSIO-21) +while keeping every historical artifact immutable. Enforce stops being a +"model earned safety certification" mode and becomes what it actually is: +a convenience mode where the user explicitly assumes the risk of model +misjudgment to reduce manual review. + +## What changes + +1. **The three promotion record gates (`cohort_qualified`, + `owner_approval`, `activation`) are no longer Enforce authority + inputs.** The truth table keeps its fail-closed health gates — audit + health, telemetry, result kind, review acknowledgement, + generation-current — but a session with no promotion records can now + hold Enforce authority. +2. **Risk contract replaces certification.** Writing `mode: "enforce"` + in config v2 *is* the consent: the user authorizes the selected judge + model to approve ordinary operations on their behalf and accepts + missed/wrong judgments. The project makes no "model safety + certification" claim and does not attempt to cover all dangerous shell + behavior. AI `allow` can auto-run; AI `deny`/`defer` still fall back + to the human dialog (no auto-deny authority is introduced). +3. **Narrow built-in high-risk override (code-level, not a sandbox).** + A small deterministic classifier recognizes clear-cut command shapes in + four categories — data loss / history rewrite (ADR 0007's enumerated + shapes), publish/deploy/infrastructure destruction, privilege + escalation / system modification, and direct credential + read/output/delete/replace. In Shadow it still calls the model + (quality observation stays) and records the override; in Enforce it + skips the model and defers immediately with an explicit risk code. + It matches only well-known explicit shapes — no alias resolution, + script-content analysis, variable expansion, or "every opaque command + defers" rule. There is deliberately **no `alwaysPrompt` user + config**: that was rejected as re-inventing policy text-matching; if a + "this permission rule always forces a human" need ever emerges, it + belongs in the permission-system's own `force_prompt` / + delegation-exclusion semantics, not in the Judge. +4. **Config v2 with explicit migration.** `version: 2` adds + `model: {provider, id}` — a fixed judge model, resolved per ask; if it + cannot be resolved (unknown model, unauthenticated, unsupported API) + the ask defers with a visible infrastructure code, never silently + falling back to the session model. Unset `model` keeps following the + session model (legacy behavior). Legacy v1 `mode: "enforce"` meant + "exact identity certified" under the old governance; upgrading must + not silently widen that consent, so a config without `version: 2` + that sets `enforce` falls back to Shadow with a diagnostic requiring + one explicit migration. +5. **One session-start notification** in Enforce mode shows the actual + judge model and the risk contract; it does not repeat per ask. + +## What is explicitly preserved + +- **Historical evidence immutability.** `promotion-records.jsonl`, the + cohort declarations/reports under `docs/testing/`, and every archived + analyzer artifact stay byte-for-byte as they are. The runtime stops + *consuming* them; nobody deletes or rewrites them. The failed cohorts + are not reopened. +- **ADR 0006 self-sufficiency stands.** The Judge-owned audit log, its + sticky health gate, and the fail-closed truth-table discipline are + unchanged; only the three promotion inputs leave the table. +- **ADR 0007's boundary is unchanged in substance** — irreversible + operations always defer. v4 keeps it at the prompt level; this ADR + adds a deterministic preflight backstop for the enumerated shapes + (exactly the fallback 0007 anticipated: "deterministic preflight defer + for the enumerated irreversible shapes"). + +## Why + +The certification framing asked a personal convenience tool to carry an +enterprise-grade evidentiary apparatus, and the apparatus could not even +carry itself: v4 exposed a protocol contradiction — the declaration froze +`earliest report asOf 22:05Z` while the report's actual `asOf` is +`21:15:06Z`, the PASS outcome was recorded at `21:20Z` (before the frozen +earliest report), and the `-04` declaration incorporates `-02`'s frozen +sections "by reference" although the current file no longer contains them. +These are recorded here as **legacy-mechanism limitations**, not rewritten +away. The honest conclusion is that the runtime gate derived no real +safety from a process this fragile, while the user-visible value of +Enforce (fewer dialogs on ordinary commands) never needed it. What +actually needs to stay certain — irreversible shapes reach a human, +auditability, fail-closed behavior on infra failures — is kept by the +narrow override and the retained health gates. + +## Consequences + +- `promotion.ts`, its tests, and `tools/promotion-record.ts` are deleted; + disk records survive. Future model-quality evidence (advisory catalog, + replay runs) informs *recommendations* and documentation only, never + runtime authority — milestone 2 of PIEXTENSIO-23. +- Rollback is `mode: "shadow"`. +- Revisit trigger: multi-user / delegated deployments where "the config + author is the risk bearer" stops being true. diff --git a/docs/testing/cohort-v4-declaration.md b/docs/testing/cohort-v4-declaration.md index beef1de..6f9a8a4 100644 --- a/docs/testing/cohort-v4-declaration.md +++ b/docs/testing/cohort-v4-declaration.md @@ -1,4 +1,14 @@ +> **Status note (2026-08-21):** superseded as a runtime governance +> mechanism by [ADR 0008](../adr/0008-enforce-user-assumed-risk.md) +> (PIEXTENSIO-23) — Enforce authority no longer consumes promotion records +> or cohort qualification. This document is preserved unchanged as +> historical audit material. It also carries a recorded protocol +> contradiction (frozen `earliest report asOf 22:05Z` vs the report's +> `asOf 21:15:06Z` and a PASS recorded at `21:20Z`; `-02` frozen sections +> incorporated by reference are no longer in this file) — noted in ADR 0008 +> as a legacy-mechanism limitation, not rewritten. + ## Outcome of `piextensio-22-v4-gpt56sol-20260820-03`: FAIL (activity floor), 87 enrollments Recorded `2026-08-20T18:50:00Z`, after the window closed at `13:20:00Z`. diff --git a/docs/testing/cohort-v4-report.md b/docs/testing/cohort-v4-report.md index 5c4aecf..ce3a9ed 100644 --- a/docs/testing/cohort-v4-report.md +++ b/docs/testing/cohort-v4-report.md @@ -1,3 +1,10 @@ +> **Status note (2026-08-21):** superseded as a runtime governance +> mechanism by [ADR 0008](../adr/0008-enforce-user-assumed-risk.md) +> (PIEXTENSIO-23) — Enforce authority no longer consumes cohort +> qualification. This report is preserved unchanged as historical audit +> material; its results speak to observed model quality, not to any +> certification of safety. + # PIEXTENSIO-22 v4 Enforce promotion cohort report ## Decision diff --git a/packages/pi-permission-ai-judge/README.md b/packages/pi-permission-ai-judge/README.md index e9c256a..5fbf7ea 100644 --- a/packages/pi-permission-ai-judge/README.md +++ b/packages/pi-permission-ai-judge/README.md @@ -1,14 +1,14 @@ # pi-permission-ai-judge -[`pi`](https://github.com/earendil-works/pi-coding-agent) 的 Bash 权限 AI 判官——对每一条进入权限对话框的命令,用会话模型给出第二意见。 +[`pi`](https://github.com/earendil-works/pi-coding-agent) 的 Bash 权限 AI 判官——对每一条进入权限对话框的命令,用 AI 模型给出第二意见。 - **影子模式**(默认):判官并行给出"允许 / 拒绝 / 交给人类"的判决,人类照常决策;判决与证据元数据全部落盘,可离线分析 -- **强制模式**:仅允许委托——仅当判决为"允许"且全部治理门通过时跳过人类对话框;任何不确定(交人类 / 拒绝 / 超时 / 健康异常)一律回落对话框。不可逆操作(如 `git clean -xfd`)永远交给人类,详见 ADR 0007 +- **强制模式**(config v2):**用户自担风险的便利模式**——判官判决"允许"且运行时健康门全部通过时跳过人类对话框;判官"拒绝/交人类/超时/健康异常"一律回落对话框,不引入自动拒绝。明确的高风险命令形状(不可逆删除、发布/基础设施销毁、提权/系统修改、直接读删凭据)永远交人类。详见 ADR 0008 ## 安装 ```bash -pi install github.com/Sikongjueluo/pi-extensions +pi install github.com/SikongJueluo/pi-extensions ``` 依赖项目启用 [`@gotgenes/pi-permission-system`](https://github.com/gotgenes/pi-permission-system) ≥ 25.4,并配置授权链(仅 UI 会话生效): @@ -23,27 +23,30 @@ pi install github.com/Sikongjueluo/pi-extensions `~/.pi/agent/pi-permission-ai-judge.config.json`,会话启动时读取: ```json -{ "mode": "shadow", "timeoutMs": 30000 } +{ + "version": 2, + "mode": "enforce", + "model": { "provider": "openai-codex", "id": "gpt-5.6-sol" }, + "timeoutMs": 30000 +} ``` -- `mode`:`shadow`(影子)| `enforce`(强制),默认影子;非法值一律回退影子 -- `timeoutMs`:5000–30000,默认 15000。**非默认超时是独立的配置群组**,不继承其他群组的晋升记录 +- `version`:配置协议版本。**旧 v1 配置(无 `version` 字段)里的 `mode: "enforce"` 不会静默获得新授权**——会回退 shadow 并提示迁移;写入 `"version": 2` 即完成一次显式迁移,这个动作本身就是风险同意 +- `mode`:`shadow`(默认)| `enforce`(强制)。非法值一律回退 shadow +- `model`(可选,仅 v2):固定判官模型,不随会话模型切换。配置的模型不存在、无认证或 API 不支持时,该次询问记为基础设施失败并交人类,**绝不静默改用会话模型**;未配置时跟随当前会话模型 +- `timeoutMs`:5000–30000,默认 15000 -## 强制模式晋升（仅仓库所有者；普通用户无需任何操作） +强制模式每次会话启动时弹一次非阻塞通知,显示实际判官模型与风险契约;不逐次重复。 -默认影子模式对使用者零门槛。强制模式意味着“判官说允许就跳过对话框”，因此启用前需要证据链：影子模式下收集一批真实流量（群组，如 110 条）证明零误放，然后所有者显式记录三次——群组合格、批准、激活（三个独立动作，精确绑定候选身份：模型 × 提示词版本 × 超时群组等 9 个字段）。任一缺失或身份漂移即自动关闸。 +## 强制模式 = 风险契约(不是安全认证) -**使用者两种选择：**信任本仓库已归档的群组证据，直接拷贝 `promotion-records.jsonl` 并配置 `mode: "enforce"`；或换用自己的模型，重新收集群组后自行记录。所有者记录命令： +手写 `mode: "enforce"` 即表示:你授权所选判官模型代为批准普通操作,并**自行承担漏判/误判风险**(ADR 0008)。本包不承诺任何"模型已获安全认证",也不试图覆盖所有危险 shell 行为。保留的运行时保护: -```bash -cd packages/pi-permission-ai-judge -npx tsx tools/promotion-record.ts --kind cohort_qualified \ - --provider openai-codex --model gpt-5.6-sol --api openai-codex-responses \ - --timeout-cohort 30000 --basis "cohort id; report path" -# 再依次 --kind owner_approval、--kind activation(三个独立显式动作) -``` +- **健康门**(全部 fail-closed):审计日志健康、遥测健康、判决结果类型、审计回执、会话世代——任一失败即回退对话框 +- **内置高风险 override**(窄范围代码级规则,不是 sandbox):明确的数据丢失/历史重写(`git clean -xfd`、`git reset --hard`、`git push --force`、`rm -rf ~` 等)、发布/部署/基础设施销毁(`npm publish`、`terraform destroy` 等)、提权/系统修改(`sudo`、`mkfs`、`dd of=/dev/*`、`shutdown` 等)、直接凭据读取/输出/删除(`cat ~/.ssh/id_*`、`~/.aws/credentials`、`~/.gnupg` 等)。Enforce 命中时不调模型、立即交人类;Shadow 命中时仍调模型(保留质量观测)并记录 override,最终照常人类决策。不解析别名/脚本内容/变量展开,没有 `alwaysPrompt` 配置 +- 回滚 = 配置切回 `shadow` -记录写入 `~/.pi/agent/extensions/pi-permission-ai-judge/promotion-records.jsonl`(只追加)。回滚 = 配置切回 `shadow`;记录保留作审计轨迹。 +历史治理(v4 及更早的 promotion cohort、三重记录门)已被 ADR 0008 取代,运行时不再读取 `promotion-records.jsonl`;旧记录与 cohort 报告原样保留作审计资料,见 `docs/testing/`。 ## 日志与工具 @@ -54,5 +57,6 @@ npx tsx tools/promotion-record.ts --kind cohort_qualified \ ## 文档 -- 治理与晋升底线:PIEXTENSIO-10;v3 失败与 v4 修复:PIEXTENSIO-19/20/22,见 `docs/testing/` -- ADR 0006(审计自持)、0007(不可逆边界):见 `docs/adr/` +- 风险契约与治理变更:ADR 0008 / PIEXTENSIO-23 +- 审计自持:ADR 0006;不可逆边界:ADR 0007,见 `docs/adr/` +- 旧 cohort 证据(v4 前含晋升治理):`docs/testing/` diff --git a/packages/pi-permission-ai-judge/src/config.ts b/packages/pi-permission-ai-judge/src/config.ts index 9dabf8f..a401531 100644 --- a/packages/pi-permission-ai-judge/src/config.ts +++ b/packages/pi-permission-ai-judge/src/config.ts @@ -4,21 +4,32 @@ import { DEFAULT_TIMEOUT_MS, MAX_TIMEOUT_MS, MIN_TIMEOUT_MS } from "./model"; /** * Judge configuration (PIEXTENSIO-3 Config acceptance, PIEXTENSIO-11 - * configuration contract). + * configuration contract; version 2 — PIEXTENSIO-23 / ADR 0008). * - * The single user-configurable field is `timeoutMs` because acceptable - * interactive wait varies by deployment. Mode is read but v0.1 authority is - * mechanically fail-closed: `enforce` loads but the judge truth table - * (PIEXTENSIO-12) can never produce real authority until the promotion - * gates exist upstream, so it always defers. + * Version 2 adds `model: {provider, id}` — an explicit fixed judge model + * — and makes hand-written `mode: "enforce"` the user's risk consent. A + * legacy (version 1 / unversioned) `mode: "enforce"` meant "exact + * identity certified" under the retired promotion governance; it must not + * silently gain the new, broader consent, so it fails closed to shadow + * with a diagnostic requiring one explicit migration to `version: 2`. * - * Global config path only — no project/env override, matching the trusted - * user-global Judge configuration of PIEXTENSIO-10. + * Everything invalid fails closed to the documented defaults + * (PIEXTENSIO-11) and records a diagnostic. Global config path only — no + * project/env override, matching the trusted user-global Judge + * configuration. */ export type JudgeMode = "shadow" | "enforce"; +/** A user-selected fixed judge model (config v2). */ +export interface JudgeModelSelection { + readonly provider: string; + readonly id: string; +} + export interface EffectiveJudgeConfig { + /** Config schema version the effective values were loaded under. */ + readonly configVersion: 1 | 2; readonly mode: JudgeMode; readonly timeoutMs: number; /** @@ -27,6 +38,8 @@ export interface EffectiveJudgeConfig { * (PIEXTENSIO-11). `default` marks the calibrated default. */ readonly timeoutCohort: "default" | number; + /** Fixed judge model (v2 only); undefined follows the session model. */ + readonly judgeModel: JudgeModelSelection | undefined; /** Validation diagnostics for the loaded raw file, newest wins per key. */ readonly diagnostics: readonly ConfigDiagnostic[]; } @@ -46,17 +59,44 @@ export interface ConfigLoadDeps { const CONFIG_FILENAME = "pi-permission-ai-judge.config.json"; const DEFAULT_CONFIG: EffectiveJudgeConfig = { + configVersion: 1, mode: "shadow", timeoutMs: DEFAULT_TIMEOUT_MS, timeoutCohort: "default", + judgeModel: undefined, diagnostics: [], }; +function fileDefaults(problem: string): EffectiveJudgeConfig { + return { + ...DEFAULT_CONFIG, + diagnostics: [{ key: "file", problem, fallback: "all defaults" }], + }; +} + +function isNonEmptyString(value: unknown): value is string { + return typeof value === "string" && value.trim().length > 0; +} + +function parseJudgeModel(value: unknown): JudgeModelSelection | undefined { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return undefined; + } + const record = value as Record; + if (!isNonEmptyString(record.provider) || !isNonEmptyString(record.id)) { + return undefined; + } + return { + provider: record.provider.trim(), + id: record.id.trim(), + }; +} + /** - * Load and validate the global config. Missing file, malformed JSON, - * unknown mode, or out-of-range timeout all fail closed to the documented - * defaults (PIEXTENSIO-11: "invalid values fail closed to the documented - * default rather than becoming unbounded") and record a diagnostic. + * Load and validate the global config. Missing file, malformed JSON, or + * an unknown version resolve to the documented defaults with one + * diagnostic. Version 1 / unversioned files keep v1 semantics except + * that `enforce` fails closed to shadow pending explicit migration. */ export function loadJudgeConfig( deps: ConfigLoadDeps, @@ -67,51 +107,49 @@ export function loadJudgeConfig( try { raw = read(path); } catch (error) { - return { - ...DEFAULT_CONFIG, - diagnostics: [ - { - key: "file", - problem: `config not readable at ${path}: ${error instanceof Error ? error.message : String(error)}`, - fallback: "all defaults", - }, - ], - }; + return fileDefaults( + `config not readable at ${path}: ${error instanceof Error ? error.message : String(error)}`, + ); } let parsed: unknown; try { parsed = JSON.parse(raw); } catch (error) { - return { - ...DEFAULT_CONFIG, - diagnostics: [ - { - key: "file", - problem: `malformed JSON: ${error instanceof Error ? error.message : String(error)}`, - fallback: "all defaults", - }, - ], - }; + return fileDefaults( + `malformed JSON: ${error instanceof Error ? error.message : String(error)}`, + ); } if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) { - return { - ...DEFAULT_CONFIG, - diagnostics: [ - { - key: "file", - problem: "top-level value is not an object", - fallback: "all defaults", - }, - ], - }; + return fileDefaults("top-level value is not an object"); } const record = parsed as Record; const diagnostics: ConfigDiagnostic[] = []; - // Mode: unknown or missing resolves to shadow (fail-closed). + // Version: unversioned files are legacy v1; anything but 1 or 2 fails + // closed to all defaults (the consent expression is uninterpretable). + let configVersion: 1 | 2 = 1; + if (record.version !== undefined) { + if (record.version === 1 || record.version === 2) { + configVersion = record.version; + } else { + return { + ...DEFAULT_CONFIG, + diagnostics: [ + { + key: "version", + problem: `unknown version ${JSON.stringify(record.version)}`, + fallback: "all defaults", + }, + ], + }; + } + } + + // Mode: unknown or missing resolves to shadow (fail-closed). A v1 + // enforce cannot silently inherit the v2 risk contract (ADR 0008). let mode: JudgeMode = "shadow"; if (record.mode !== undefined) { if (record.mode === "shadow" || record.mode === "enforce") { @@ -124,6 +162,39 @@ export function loadJudgeConfig( }); } } + if (configVersion === 1 && mode === "enforce") { + mode = "shadow"; + diagnostics.push({ + key: "mode", + problem: + "v1 enforce means certified identity under the retired promotion governance (ADR 0008); add \"version\": 2 to consent to the user-assumed-risk contract", + fallback: "shadow (v1 enforce requires explicit migration)", + }); + } + + // Judge model: v2-only. A malformed model poisons the consent file — + // fail the whole config closed to shadow rather than silently + // substituting the session model. + let judgeModel: JudgeModelSelection | undefined; + if (record.model !== undefined) { + if (configVersion === 2) { + judgeModel = parseJudgeModel(record.model); + if (judgeModel === undefined) { + mode = "shadow"; + diagnostics.push({ + key: "model", + problem: `invalid model ${JSON.stringify(record.model)} (expected {provider, id} with non-empty strings)`, + fallback: "shadow (no judge model)", + }); + } + } else { + diagnostics.push({ + key: "model", + problem: "model selection requires \"version\": 2", + fallback: "session model", + }); + } + } // Timeout: integers in [5_000, 30_000]; anything else falls back to the // documented 15,000 ms default. Boundary semantics: 4,999 and 30,001 @@ -149,5 +220,12 @@ export function loadJudgeConfig( } } - return Object.freeze({ mode, timeoutMs, timeoutCohort, diagnostics }); -} \ No newline at end of file + return Object.freeze({ + configVersion, + mode, + timeoutMs, + timeoutCohort, + judgeModel, + diagnostics, + }); +} diff --git a/packages/pi-permission-ai-judge/src/highrisk.ts b/packages/pi-permission-ai-judge/src/highrisk.ts new file mode 100644 index 0000000..65b8fe1 --- /dev/null +++ b/packages/pi-permission-ai-judge/src/highrisk.ts @@ -0,0 +1,360 @@ +/** + * Narrow built-in high-risk override (ADR 0008; PIEXTENSIO-23). + * + * A deterministic preflight that recognizes clear-cut command shapes in + * four categories and always defers them to the human dialog: + * + * - `data_loss` / `history_rewrite`: the ADR 0007 enumerated irreversible + * shapes (git clean -xfd, git reset --hard, git checkout/restore ., + * git push --force, rm -rf against home/root targets); + * - `publish_deploy`: package publishing and infrastructure destruction + * (npm/pnpm/yarn/cargo publish, terraform destroy); + * - `system_modify`: privilege escalation and system-level modification + * (sudo, mkfs, dd onto devices, shutdown/reboot/halt/poweroff); + * - `credential_access`: direct read/output/delete/replace of well-known + * credential files (~/.ssh/id_*, authorized_keys, ~/.aws/credentials, + * ~/.netrc, ~/.gnupg, gcloud application-default credentials), including + * redirect writes (`>`, `>>`, `N>`) and `tee`. + * + * Scanning is quote-, escape-, and separator-aware (`;`, `&&`, `||`, `|`, + * `&`, newlines outside quotes split units), so ordinary compound inputs + * are neither bypassed (`echo ready & npm publish`) nor false-positived + * (`printf 'x; npm publish;'`). Deliberately NOT a sandbox: only + * well-known explicit command shapes are matched. No alias resolution, + * no script-content analysis, no variable expansion, no "every opaque + * command defers" rule, and no user-facing `alwaysPrompt` config. + * Behavior: in Enforce a hit skips the model and defers immediately with + * code `high_risk_override`; in Shadow the model is still called (quality + * observation) and the override is recorded. + */ + +export type HighRiskCategory = + | "data_loss" + | "history_rewrite" + | "publish_deploy" + | "system_modify" + | "credential_access"; + +export interface HighRiskMatch { + readonly category: HighRiskCategory; + readonly rule: string; +} + +/** Characters that separate command units when unquoted and unescaped. */ +const UNIT_SEPARATORS = new Set([";", "|", "&", "\n"]); +/** Output-redirection operators, emitted as standalone tokens (`>`, `>>`, `2>`). */ +const REDIRECT_OP = /^(?:\d+)?>{1,2}$/; + +/** + * Scan a complete Bash input into units of tokens. Single- and + * double-quoted segments become one token (quotes stripped); backslash + * escapes are taken literally; separators inside quotes or escapes do + * not split. Collapsed operators (`&&`, `||`) produce empty units that + * are dropped. This is lexical hygiene only — no alias, variable, or + * substitution semantics. + */ +function scanCommand(command: string): string[][] { + const units: string[][] = []; + let tokens: string[] = []; + let current = ""; + let quote: string | null = null; + + const pushToken = (): void => { + if (current.length > 0) { + tokens.push(current); + current = ""; + } + }; + const pushUnit = (): void => { + pushToken(); + if (tokens.length > 0) { + units.push(tokens); + tokens = []; + } + }; + + for (let i = 0; i < command.length; i += 1) { + const ch = command[i]; + if (quote !== null) { + if (ch === quote) { + quote = null; + } else { + current += ch; + } + continue; + } + if (ch === '"' || ch === "'") { + pushToken(); + quote = ch; + continue; + } + if (ch === "\\") { + current += command[i + 1] ?? ""; + i += 1; + continue; + } + if (UNIT_SEPARATORS.has(ch)) { + pushUnit(); + continue; + } + if (ch === ">") { + const fd = /^\d+$/.exec(current); + pushToken(); + let op = fd === null ? ">" : `${fd[0]}>`; + if (command[i + 1] === ">") { + op += ">"; + i += 1; + } + tokens.push(op); + continue; + } + if (/\s/.test(ch)) { + pushToken(); + continue; + } + current += ch; + } + pushUnit(); + return units; +} + +/** Combined single-letter git flags, e.g. `-xfd` → has x, f, d. */ +function shortFlagLetters(token: string): Set { + const letters = new Set(); + if (!/^-[a-zA-Z]+$/.test(token)) { + return letters; + } + for (const letter of token.slice(1)) { + letters.add(letter); + } + return letters; +} + +function hasGitFlag(args: string[], letter: string, long: string): boolean { + for (const arg of args) { + if (arg === `--${long}`) { + return true; + } + if (shortFlagLetters(arg).has(letter)) { + return true; + } + } + return false; +} + +function classifyGit(args: string[]): HighRiskMatch | undefined { + const subcommand = args[0]; + const rest = args.slice(1); + switch (subcommand) { + case "clean": { + // git clean with force + ignored-file removal (-f/-x/-d shapes). + // Dry-run (-n) never matches. + const hasDryRun = rest.some((arg) => arg === "-n" || shortFlagLetters(arg).has("n")); + const force = hasGitFlag(rest, "f", "force"); + const ignored = hasGitFlag(rest, "x", "x") || rest.includes("-x"); + if (force && ignored && !hasDryRun) { + return { category: "data_loss", rule: "git_clean_forced_x" }; + } + return undefined; + } + case "reset": + if (rest.includes("--hard")) { + return { category: "data_loss", rule: "git_reset_hard" }; + } + return undefined; + case "checkout": + case "restore": { + // Whole-tree discard: `git checkout -- .` / `git checkout .` / + // `git restore .` (optionally `--worktree`-style flags first). + const positional = rest.filter((arg) => !arg.startsWith("-")); + const dotIndex = positional.lastIndexOf("."); + if (dotIndex !== -1 && positional.slice(dotIndex).every((arg) => arg === ".")) { + return { + category: "data_loss", + rule: subcommand === "checkout" ? "git_checkout_dot" : "git_restore_dot", + }; + } + return undefined; + } + case "push": { + const forced = + rest.includes("--force") || + rest.includes("-f") || + rest.includes("--force-with-lease") || + rest.some((arg) => arg.startsWith("+") && arg.length > 1); + if (forced) { + return { category: "history_rewrite", rule: "git_push_force" }; + } + return undefined; + } + default: + return undefined; + } +} + +function classifyRm(args: string[]): HighRiskMatch | undefined { + const recursive = args.some((arg) => /^-[a-zA-Z]*[rR][a-zA-Z]*$/.test(arg) || arg === "-r" || arg === "-R" || arg === "--recursive"); + const forced = hasGitFlag(args, "f", "force"); + if (!recursive || !forced) { + return undefined; + } + const targets = args.filter((arg) => !arg.startsWith("-") && !REDIRECT_OP.test(arg)); + const dangerous = targets.some( + (target) => + target === "/" || + target === "~" || + target === "$HOME" || + target.startsWith("/*") || + target.startsWith("~/") || + target.startsWith("~") || + target.startsWith("$HOME/"), + ); + return dangerous + ? { category: "data_loss", rule: "rm_rf_home_or_root" } + : undefined; +} + +function classifyPublish(head: string, args: string[]): HighRiskMatch | undefined { + const sub = args[0]; + switch (head) { + case "npm": + case "pnpm": + case "yarn": + case "cargo": + if (sub === "publish") { + return { category: "publish_deploy", rule: `${head}_publish` }; + } + return undefined; + case "terraform": + if (sub === "destroy") { + return { category: "publish_deploy", rule: "terraform_destroy" }; + } + return undefined; + default: + return undefined; + } +} + +function classifySystem(head: string, args: string[]): HighRiskMatch | undefined { + if (head === "sudo") { + return { category: "system_modify", rule: "sudo" }; + } + if (head.startsWith("mkfs")) { + return { category: "system_modify", rule: "mkfs" }; + } + if (head === "shutdown" || head === "reboot" || head === "halt" || head === "poweroff") { + return { category: "system_modify", rule: head }; + } + if (head === "dd") { + if (args.some((arg) => /^of=\/dev\//.test(arg))) { + return { category: "system_modify", rule: "dd_to_device" }; + } + } + return undefined; +} + +/** Read/alter verbs that touch a credential path directly. */ +const CREDENTIAL_VERBS = new Set([ + "cat", + "less", + "more", + "head", + "tail", + "bat", + "zcat", + "tee", + "cp", + "mv", + "rm", + "install", + "scp", + "rsync", +]); + +const CREDENTIAL_PATH_PATTERNS: ReadonlyArray<{ pattern: RegExp; rule: string }> = [ + { pattern: /^~\/\.ssh\/id_./, rule: "ssh_private_key" }, + { pattern: /^~\/\.ssh\/authorized_keys/, rule: "ssh_authorized_keys" }, + { pattern: /^~\/\.aws\/credentials/, rule: "aws_credentials" }, + { pattern: /^~\/\.netrc/, rule: "netrc" }, + { pattern: /^~\/\.gnupg(\/|$)/, rule: "gnupg" }, + { + pattern: /^~\/\.config\/gcloud\/application_default_credentials/, + rule: "gcloud_default_credentials", + }, + { pattern: /^\$HOME\/\.ssh\/id_./, rule: "ssh_private_key" }, + { pattern: /^\$HOME\/\.ssh\/authorized_keys/, rule: "ssh_authorized_keys" }, + { pattern: /^\$HOME\/\.aws\/credentials/, rule: "aws_credentials" }, + { pattern: /^\$HOME\/\.netrc/, rule: "netrc" }, + { pattern: /^\$HOME\/\.gnupg(\/|$)/, rule: "gnupg" }, +]; + +function matchCredentialPath( + value: string, +): { rule: string } | undefined { + for (const { pattern, rule } of CREDENTIAL_PATH_PATTERNS) { + if (pattern.test(value)) { + return { rule }; + } + } + return undefined; +} + +function classifyCredential(args: string[]): HighRiskMatch | undefined { + for (const arg of args) { + if (arg.startsWith("-") || REDIRECT_OP.test(arg)) { + continue; + } + const match = matchCredentialPath(arg); + if (match !== undefined) { + return { category: "credential_access", rule: match.rule }; + } + } + return undefined; +} + +/** + * Direct credential replacement via output redirection — any verb writing + * a credential path (`>`, `>>`, fd-prefixed). Checked per unit regardless + * of the leading verb, since `echo x > ~/.aws/credentials` replaces the + * credential whatever the writer is. + */ +function classifyCredentialRedirect(tokens: string[]): HighRiskMatch | undefined { + for (let i = 0; i < tokens.length; i += 1) { + if (!REDIRECT_OP.test(tokens[i])) { + continue; + } + const target = tokens[i + 1]; + if (target === undefined) { + continue; + } + const match = matchCredentialPath(target); + if (match !== undefined) { + return { + category: "credential_access", + rule: `${match.rule}_redirect_write`, + }; + } + } + return undefined; +} + +/** + * Classify a complete Bash input against the built-in high-risk shapes. + * Any matching unit marks the whole input; the first match wins. + */ +export function classifyHighRisk(fullCommand: string): HighRiskMatch | undefined { + for (const tokens of scanCommand(fullCommand)) { + const [head, ...args] = tokens; + const match = + classifyCredentialRedirect(tokens) ?? + (CREDENTIAL_VERBS.has(head) ? classifyCredential(args) : undefined) ?? + classifySystem(head, args) ?? + (head === "git" ? classifyGit(args) : undefined) ?? + (head === "rm" ? classifyRm(args) : undefined) ?? + classifyPublish(head, args); + if (match !== undefined) { + return match; + } + } + return undefined; +} diff --git a/packages/pi-permission-ai-judge/src/index.ts b/packages/pi-permission-ai-judge/src/index.ts index 2fe6e52..d372b87 100644 --- a/packages/pi-permission-ai-judge/src/index.ts +++ b/packages/pi-permission-ai-judge/src/index.ts @@ -24,16 +24,8 @@ import { conversationProbeFromSession, type ConversationEvidence, } from "./conversation"; -import { - evaluateEnforceAuthority, - type EnforceGateState, -} from "./judge"; -import { - loadPromotionRecords, - resolvePromotionGates, - type CandidateIdentity, - type PromotionRecordsSnapshot, -} from "./promotion"; +import { classifyHighRisk, type HighRiskMatch } from "./highrisk"; +import { evaluateEnforceAuthority, type EnforceGateState } from "./judge"; const LINK_NAME = "ai-bash-judge"; const REVIEW_SCHEMA_VERSION = 1; @@ -58,15 +50,6 @@ interface RootSession { readonly getCwd: () => string; /** Judge-owned audit log (ADR 0006); unhealthy refuses Enforce authority. */ readonly auditLog: AuditLog; - /** Promotion-records snapshot loaded at session start (PIEXTENSIO-21); - * gates are resolved per ask against the live candidate identity. */ - readonly promotionRecords: PromotionRecordsSnapshot; - /** Static candidate-identity fields (everything except the model - * segment, which is captured per ask). */ - readonly identityBase: Omit< - CandidateIdentity, - "provider" | "model" | "api" - >; } const EMPTY_CONVERSATION: ConversationEvidence = { @@ -122,9 +105,6 @@ function resultBase( requestId: details.requestId, judgeRuntimeId, mode: config.mode, - // v0.1 fail-closed: `enforce` loads here but the truth table can - // never grant real authority, so the effective verdict below stays - // defer; the configured mode is recorded for audit. timeoutCohort: config.timeoutCohort, origin: details.forwarding !== undefined || @@ -137,7 +117,6 @@ function resultBase( }; } - /** Read the permission-system review-log toggle (default true when unset). */ function readPermissionReviewLogEnabled(): boolean { try { @@ -157,6 +136,30 @@ function readPermissionReviewLogEnabled(): boolean { } } +/** + * Resolve the judge model for one ask (ADR 0008 / PIEXTENSIO-23). + * + * A configured v2 `model` is a fixed judge model resolved per ask against + * the live registry. Resolution failure (unknown model, no configured + * auth) is reported as an explicit failure — never a silent fallback to + * the session model. No configured model follows the session model + * (legacy behavior). + */ +function resolveJudgeModel( + config: EffectiveJudgeConfig, + sessionModel: Model | undefined, + registry: ModelRegistry, +): { kind: "model"; model: Model | undefined; source: "configured" | "session" } | { kind: "unavailable" } { + if (config.judgeModel === undefined) { + return { kind: "model", model: sessionModel, source: "session" }; + } + const found = registry.find(config.judgeModel.provider, config.judgeModel.id); + if (found === undefined || !registry.hasConfiguredAuth(found)) { + return { kind: "unavailable" }; + } + return { kind: "model", model: found, source: "configured" }; +} + /** Register a Shadow-only structured-output judge for local native Bash asks. */ export default function permissionAiJudge(pi: ExtensionAPI): void { let root: RootSession | undefined; @@ -282,14 +285,69 @@ export default function permissionAiJudge(pi: ExtensionAPI): void { return { kind: "defer" }; } - // Per-request model capture (PIEXTENSIO-3 cat.3): an - // in-request switch does not change an in-flight attempt; - // a between-request switch affects the next attempt. The - // probe reads the live current model here, not at start. - const availability = createModelAvailability( + // Built-in high-risk override (ADR 0008): clear-cut + // irreversible/system shapes always defer. In Enforce the + // model is skipped entirely; in Shadow it still runs for + // quality observation and the override is recorded. + const risk: HighRiskMatch | undefined = classifyHighRisk( + evidence.fullCommand, + ); + if (risk !== undefined && captured.config.mode === "enforce") { + emitResult({ + ...resultBase( + captured.judgeRuntimeId, + details, + startedAt, + captured.config, + ), + resultKind: "preflight_defer", + verdict: null, + effectiveVerdict: "defer", + modelCalled: false, + code: "high_risk_override", + riskCategory: risk.category, + riskRule: risk.rule, + evidenceQuality: evidenceQuality(true, EMPTY_CONVERSATION, captured.getCwd()), + }); + return { kind: "defer" }; + } + + // Per-request judge-model resolution (PIEXTENSIO-3 cat.3 + // for the session model; ADR 0008 for a configured fixed + // model). A configured model that cannot be resolved is + // an observable infrastructure failure — never a silent + // fallback to the session model. + const resolved = resolveJudgeModel( + captured.config, captured.getModel(), captured.modelRegistry, ); + if (resolved.kind === "unavailable") { + emitResult({ + ...resultBase( + captured.judgeRuntimeId, + details, + startedAt, + captured.config, + ), + resultKind: "infrastructure_failure", + verdict: null, + effectiveVerdict: "defer", + modelCalled: false, + code: "judge_model_unavailable", + provider: captured.config.judgeModel?.provider ?? null, + model: captured.config.judgeModel?.id ?? null, + api: null, + riskOverride: risk ?? null, + evidenceQuality: evidenceQuality(true, EMPTY_CONVERSATION, captured.getCwd()), + }); + return { kind: "defer" }; + } + const modelSource = resolved.source; + const availability: ModelAvailability = createModelAvailability( + resolved.model, + captured.modelRegistry, + ); // Conversation evidence is captured at ask time from the // live serving branch, not at session start: the newest // user intent is the ask's intent. @@ -303,36 +361,19 @@ export default function permissionAiJudge(pi: ExtensionAPI): void { conversation, ); - // Enforce truth table (PIEXTENSIO-3 cat.4 / M5): the - // promotion gates resolve per ask from the session-start - // records snapshot against the live candidate identity - // (PIEXTENSIO-21) — the current model segment is part of - // that identity, so a mid-session model switch cannot - // inherit another segment's records, and identity drift - // after recording fails closed. The truth table is the - // single authority seam: the owner's records flip the - // gate inputs, never the callback's return path. - // reviewAcknowledged is true in the ADR 0006 sense: the - // Judge-owned audit write for this result happens before - // the authority return, and a failed write flips - // auditHealthy sticky-unhealthy, closing authority for - // every later ask. - const liveIdentity: CandidateIdentity = { - ...captured.identityBase, - provider: result.metadata?.provider ?? "", - model: result.metadata?.model ?? "", - api: result.metadata?.api ?? "", - }; - const gates = resolvePromotionGates( - captured.promotionRecords, - liveIdentity, - ); + // Enforce truth table (PIEXTENSIO-3 cat.4 / M5; ADR 0008): + // the fail-closed runtime health gates — audit health, + // telemetry, result kind, verdict, review + // acknowledgement, generation currency — are the single + // authority seam; the retired promotion gates are no + // longer inputs. reviewAcknowledged is true in the ADR + // 0006 sense: the Judge-owned audit write for this result + // happens before the authority return, and a failed write + // flips auditHealthy sticky-unhealthy, closing authority + // for every later ask. const gateState: EnforceGateState = { auditHealthy: captured.auditLog.healthy(), telemetryHealth: sink.health(), - cohortQualified: gates.cohortQualified, - ownerApprovalRecorded: gates.ownerApprovalRecorded, - activationRecorded: gates.activationRecorded, resultKind: result.kind === "judgment" ? "judgment" @@ -363,9 +404,11 @@ export default function permissionAiJudge(pi: ExtensionAPI): void { authorityBlockedBy, modelCalled: true, code: null, + modelSource, provider: result.metadata.provider, model: result.metadata.model, api: result.metadata.api, + riskOverride: risk ?? null, // Log keys deliberately avoid the substring // "token": permission-system masks any key matching // /token/i (structural key-name redaction), which @@ -390,9 +433,11 @@ export default function permissionAiJudge(pi: ExtensionAPI): void { authorityBlockedBy, modelCalled: result.modelCalled, code: result.code, + modelSource, provider: result.metadata?.provider ?? null, model: result.metadata?.model ?? null, api: result.metadata?.api ?? null, + riskOverride: risk ?? null, inputUsage: result.inputTokens ?? null, outputUsage: result.outputTokens ?? null, modelLatencyMs: result.modelLatencyMs, @@ -426,26 +471,6 @@ export default function permissionAiJudge(pi: ExtensionAPI): void { const runtimeId = crypto.randomUUID(); const config = loadJudgeConfig({ agentDir: getAgentDir() }); - // Static candidate-identity fields. The judge and permission-system - // versions must track package.json / the peer floor; the cohort - // declaration used exactly these values. - const identityBase = { - judge: "@sikongjueluo/pi-permission-ai-judge@0.0.1", - permissionSystem: "25.4.0", - promptVersion: PROMPT_VERSION, - toolSchemaVersion: TOOL_SCHEMA_VERSION, - reviewSchemaVersion: String(REVIEW_SCHEMA_VERSION), - timeoutCohort: config.timeoutCohort, - }; - const promotionRecords = loadPromotionRecords({ - agentDir: getAgentDir(), - }); - if (promotionRecords.diagnostic !== null) { - ctx.ui.notify( - `ai-bash-judge promotion records: ${promotionRecords.diagnostic}; Enforce gates stay closed`, - "warning", - ); - } root = { getSessionId: () => ctx.sessionManager.getSessionId(), expectedSessionId: sessionId, @@ -459,8 +484,6 @@ export default function permissionAiJudge(pi: ExtensionAPI): void { agentDir: getAgentDir(), runtimeId, }), - promotionRecords, - identityBase, conversation: conversationProbeFromSession(ctx.sessionManager), getCwd: () => ctx.sessionManager.getCwd(), }; @@ -470,6 +493,25 @@ export default function permissionAiJudge(pi: ExtensionAPI): void { "warning", ); } + // One non-blocking session notice in Enforce mode: the risk + // contract and the effective judge model (ADR 0008). Not repeated + // per ask. + if (root.config.mode === "enforce") { + const configured = root.config.judgeModel; + const judgeModelDescription = + configured !== undefined + ? `${configured.provider}/${configured.id} (configured)` + : (() => { + const sessionModel = ctx.model; + return sessionModel === undefined + ? "the current session model (none resolved yet)" + : `${sessionModel.provider}/${sessionModel.id} (current session model)`; + })(); + ctx.ui.notify( + `ai-bash-judge Enforce active: ${judgeModelDescription} judges Bash asks; allow skips the dialog — you accept the risk of model misjudgment (ADR 0008). High-risk shapes (irreversible, publish, system, credentials) always ask.`, + "info", + ); + } tryRegister(); }); diff --git a/packages/pi-permission-ai-judge/src/judge.ts b/packages/pi-permission-ai-judge/src/judge.ts index 2b95a7c..1b1afe9 100644 --- a/packages/pi-permission-ai-judge/src/judge.ts +++ b/packages/pi-permission-ai-judge/src/judge.ts @@ -1,13 +1,14 @@ import type { TelemetryHealth } from "./review"; /** - * Enforce truth table (PIEXTENSIO-3 cat.4 / M5). + * Enforce truth table (PIEXTENSIO-3 cat.4 / M5; ADR 0008 / PIEXTENSIO-23). * * `allow` authority requires every gate to hold **independently**; any one - * false forces defer. Since PIEXTENSIO-21 the promotion gates read the - * Judge-owned records file (exact-identity qualification, fail-closed); - * with no records, every mode mechanically defers — verified by the - * truth-table tests toggling each condition. + * false forces defer. Since ADR 0008 the promotion gates (cohort + * qualification, owner approval, activation) are no longer authority + * inputs: hand-written `mode: "enforce"` in config v2 is the user's risk + * consent, and the remaining gates are the fail-closed runtime health + * checks — verified by the truth-table tests toggling each condition. */ export type EnforceGateState = { @@ -15,12 +16,6 @@ export type EnforceGateState = { readonly auditHealthy: boolean; /** Runtime telemetry healthy at decision time. */ readonly telemetryHealth: TelemetryHealth; - /** Qualified passing promotion cohort (PIEXTENSIO-10 floor). */ - readonly cohortQualified: boolean; - /** Recorded owner approval for the exact candidate identity. */ - readonly ownerApprovalRecorded: boolean; - /** Independent explicit activation act (distinct from approval). */ - readonly activationRecorded: boolean; /** The attempt's terminal result kind. */ readonly resultKind: "judgment" | "preflight_defer" | "infrastructure_failure"; /** The semantic verdict when resultKind is judgment, else null. */ @@ -52,15 +47,6 @@ export function evaluateEnforceAuthority( if (state.telemetryHealth !== "healthy") { return { kind: "defer", blockedBy: `telemetry_${state.telemetryHealth}` }; } - if (!state.cohortQualified) { - return { kind: "defer", blockedBy: "cohort_not_qualified" }; - } - if (!state.ownerApprovalRecorded) { - return { kind: "defer", blockedBy: "owner_approval_absent" }; - } - if (!state.activationRecorded) { - return { kind: "defer", blockedBy: "activation_absent" }; - } if (state.resultKind !== "judgment") { return { kind: "defer", blockedBy: `result_${state.resultKind}` }; } diff --git a/packages/pi-permission-ai-judge/src/model.ts b/packages/pi-permission-ai-judge/src/model.ts index 8811d4a..1923cca 100644 --- a/packages/pi-permission-ai-judge/src/model.ts +++ b/packages/pi-permission-ai-judge/src/model.ts @@ -28,6 +28,7 @@ const MAX_OUTPUT_TOKENS = 4_096; export type InfrastructureCode = | "no_model" | "unsupported_api" + | "judge_model_unavailable" | "timeout" | "aborted" | "model_error" diff --git a/packages/pi-permission-ai-judge/src/promotion.ts b/packages/pi-permission-ai-judge/src/promotion.ts deleted file mode 100644 index ed48bc2..0000000 --- a/packages/pi-permission-ai-judge/src/promotion.ts +++ /dev/null @@ -1,322 +0,0 @@ -import { - closeSync, - existsSync, - mkdirSync, - openSync, - readFileSync, - writeSync, - fsyncSync, -} from "node:fs"; -import { join } from "node:path"; -import { stripForbiddenKeys } from "./review"; - -/** - * Judge-owned promotion-gate records (ADR 0006 self-sufficiency; PIEXTENSIO-10 - * promotion governance; PIEXTENSIO-21 seam). - * - * The Enforce truth table consumes three gate inputs — cohort qualification, - * owner approval, activation — that v0.1 hardcoded to false. This module is - * their storage and resolution: - * - * - **Records file**: append-only JSONL under the agent dir, fsync per - * record, same discipline as the audit log. Three record kinds: - * `cohort_qualified`, `owner_approval`, `activation`. Owner actions write - * records offline (the CLI helper in tools/); the online judge only reads. - * - **Exact-identity qualification**: a record qualifies the gate only if - * its candidate identity matches the live identity field-for-field. A - * record for any other identity is inert. This is the PIEXTENSIO-10 rule - * that approval binds to the exact candidate, and it makes promotion - * immune to identity drift after the records are written. - * - **Fail-closed everywhere**: missing file, unreadable file, malformed - * line, or shape-invalid record ⇒ the gate stays false. No record kind - * may flip any gate other than its own. Gates are resolved once at - * session start (immutable snapshot for the session), mirroring the - * reload-only config contract. - * - * Mode is intentionally *not* part of a candidate identity: `mode` is the - * authority knob (shadow/enforce), while identity is what the records - * certify. Requiring mode equality would let a shadow cohort qualify an - * enforce activation record — or vice versa — which the governance - * explicitly separates. - */ - -/** The candidate-identity fields a promotion record must match exactly. */ -export interface CandidateIdentity { - readonly judge: string; - readonly permissionSystem: string; - readonly provider: string; - readonly model: string; - readonly api: string; - readonly promptVersion: string; - readonly toolSchemaVersion: string; - readonly reviewSchemaVersion: string; - readonly timeoutCohort: "default" | number; -} - -export type PromotionRecordKind = - | "cohort_qualified" - | "owner_approval" - | "activation"; - -export interface PromotionRecord { - readonly kind: PromotionRecordKind; - readonly candidateIdentity: CandidateIdentity; - /** ISO timestamp written by the record author. */ - readonly recordedAt: string; - /** Human-readable basis (cohort id, report reference, approval note). */ - readonly basis: string; -} - -export interface PromotionRecordsSnapshot { - /** Shape-valid records parsed from the file (all identities). */ - readonly records: readonly PromotionRecord[]; - /** False when the file exists but is unreadable or has malformed lines. */ - readonly healthy: boolean; - readonly diagnostic: string | null; - /** Absolute path of the records file (empty string when unset). */ - readonly path: string; -} - -export interface PromotionRecordsDeps { - /** User-global agent dir (`~/.pi/agent`). */ - readonly agentDir: string; - /** Injectable reader for tests; defaults to readFileSync(utf-8). */ - readonly readFile?: (path: string) => string; -} - -const RECORDS_DIR_SEGMENTS = ["extensions", "pi-permission-ai-judge"]; -const RECORDS_FILENAME = "promotion-records.jsonl"; - -const IDENTITY_KEYS = [ - "judge", - "permissionSystem", - "provider", - "model", - "api", - "promptVersion", - "toolSchemaVersion", - "reviewSchemaVersion", - "timeoutCohort", -] as const; - -const STRING_IDENTITY_KEYS = IDENTITY_KEYS.filter( - (key) => key !== "timeoutCohort", -); - -export function promotionRecordsPath(agentDir: string): string { - return join(agentDir, ...RECORDS_DIR_SEGMENTS, RECORDS_FILENAME); -} - -function identityMatches( - record: CandidateIdentity, - live: CandidateIdentity, -): boolean { - return IDENTITY_KEYS.every((key) => record[key] === live[key]); -} - -function isRecordShape(value: unknown): value is PromotionRecord { - if (typeof value !== "object" || value === null || Array.isArray(value)) { - return false; - } - const record = value as Record; - if ( - record.kind !== "cohort_qualified" && - record.kind !== "owner_approval" && - record.kind !== "activation" - ) { - return false; - } - if ( - typeof record.recordedAt !== "string" || - record.recordedAt.length === 0 - ) { - return false; - } - if (typeof record.basis !== "string") { - return false; - } - const identity = record.candidateIdentity; - if ( - typeof identity !== "object" || - identity === null || - Array.isArray(identity) - ) { - return false; - } - const identityRecord = identity as Record; - // Every field except timeoutCohort is a string; timeoutCohort is - // "default" or a positive integer (config-cohort semantics). - const timeoutCohort = identityRecord.timeoutCohort; - const validTimeoutCohort = - timeoutCohort === "default" || - (typeof timeoutCohort === "number" && - Number.isInteger(timeoutCohort) && - timeoutCohort > 0); - if ( - !STRING_IDENTITY_KEYS.every((key) => - typeof identityRecord[key] === "string", - ) || - !validTimeoutCohort - ) { - return false; - } - return true; -} - -/** - * Load and parse the records file once per session (immutable snapshot). - * - * Read-only and side-effect free. Malformed or partial state never raises — - * the snapshot reports `healthy: false`, which `resolvePromotionGates` - * turns into all-gates-closed. A missing file is the normal pre-promotion - * state: healthy with zero records. - */ -export function loadPromotionRecords( - deps: PromotionRecordsDeps, -): PromotionRecordsSnapshot { - const file = promotionRecordsPath(deps.agentDir); - const read = deps.readFile ?? ((p: string) => readFileSync(p, "utf-8")); - - let raw: string; - try { - raw = read(file); - } catch { - return { - records: [], - healthy: true, - diagnostic: existsSync(file) ? `records unreadable at ${file}` : null, - path: file, - }; - } - - const records: PromotionRecord[] = []; - let malformed = false; - for (const line of raw.split("\n")) { - const trimmed = line.trim(); - if (trimmed.length === 0) { - continue; - } - let parsed: unknown; - try { - parsed = JSON.parse(trimmed); - } catch { - malformed = true; - continue; - } - if (!isRecordShape(parsed)) { - malformed = true; - continue; - } - records.push(parsed); - } - - if (malformed) { - return { - records: [], - healthy: false, - diagnostic: `malformed records at ${file}`, - path: file, - }; - } - return { records, healthy: true, diagnostic: null, path: file }; -} - -/** The three Enforce promotion gates for one live candidate identity. */ -export interface PromotionGateResolution { - readonly cohortQualified: boolean; - readonly ownerApprovalRecorded: boolean; - readonly activationRecorded: boolean; -} - -const ALL_GATES_CLOSED: PromotionGateResolution = { - cohortQualified: false, - ownerApprovalRecorded: false, - activationRecorded: false, -}; - -/** - * Resolve the three Enforce promotion gates for the live candidate - * identity against a session-start records snapshot. - * - * Called per ask: the live identity carries the current model segment, so - * a mid-session model switch cannot inherit another segment's records. - * Unhealthy snapshot ⇒ all gates closed (fail-closed). A record qualifies - * only when its candidate identity matches field-for-field; records for - * any other identity are inert. - */ -export function resolvePromotionGates( - snapshot: PromotionRecordsSnapshot, - liveIdentity: CandidateIdentity, -): PromotionGateResolution { - if (!snapshot.healthy) { - return ALL_GATES_CLOSED; - } - let cohortQualified = false; - let ownerApprovalRecorded = false; - let activationRecorded = false; - for (const record of snapshot.records) { - if (!identityMatches(record.candidateIdentity, liveIdentity)) { - continue; - } - if (record.kind === "cohort_qualified") cohortQualified = true; - if (record.kind === "owner_approval") ownerApprovalRecorded = true; - if (record.kind === "activation") activationRecorded = true; - } - return { cohortQualified, ownerApprovalRecorded, activationRecorded }; -} - -export interface AppendPromotionRecordDeps { - /** User-global agent dir (`~/.pi/agent`). */ - readonly agentDir: string; - readonly record: PromotionRecord; - /** Injectable clock for tests; defaults to ISO-now. */ - readonly now?: () => string; -} - -/** - * Offline owner action: append one promotion record (append + fsync, same - * write discipline as the audit log). Never imported by online modules — - * the CLI helper in tools/ is the only in-package consumer. - * - * Returns null on success or a human-readable failure reason. - */ -export function appendPromotionRecord( - deps: AppendPromotionRecordDeps, -): string | null { - const now = deps.now ?? (() => new Date().toISOString()); - const dir = join(deps.agentDir, ...RECORDS_DIR_SEGMENTS); - const file = join(dir, RECORDS_FILENAME); - if (!isRecordShape(deps.record)) { - return "record is not shape-valid"; - } - try { - mkdirSync(dir, { recursive: true }); - } catch (error) { - return `cannot create ${dir}: ${error instanceof Error ? error.message : String(error)}`; - } - const record = JSON.stringify({ - recordedAt: now(), - ...stripForbiddenKeys({ - kind: deps.record.kind, - candidateIdentity: deps.record.candidateIdentity, - basis: deps.record.basis, - } as Record), - }); - let fd: number | undefined; - try { - fd = openSync(file, "a"); - writeSync(fd, `${record}\n`); - fsyncSync(fd); - return null; - } catch (error) { - return `cannot append to ${file}: ${error instanceof Error ? error.message : String(error)}`; - } finally { - if (fd !== undefined) { - try { - closeSync(fd); - } catch { - // Close failure does not un-fail the append. - } - } - } -} diff --git a/packages/pi-permission-ai-judge/test/config.test.ts b/packages/pi-permission-ai-judge/test/config.test.ts index a5cb2e0..4523474 100644 --- a/packages/pi-permission-ai-judge/test/config.test.ts +++ b/packages/pi-permission-ai-judge/test/config.test.ts @@ -20,9 +20,11 @@ describe("loadJudgeConfig — missing and malformed", () => { it("resolves a missing file to all defaults with one diagnostic", () => { const config = loadJudgeConfig(deps()); expect(config).toEqual({ + configVersion: 1, mode: "shadow", timeoutMs: 15_000, timeoutCohort: "default", + judgeModel: undefined, diagnostics: [ expect.objectContaining({ key: "file", fallback: "all defaults" }), ], @@ -45,19 +47,76 @@ describe("loadJudgeConfig — missing and malformed", () => { }); }); +describe("loadJudgeConfig — version selection", () => { + it("treats a missing version as v1", () => { + const config = loadJudgeConfig(deps({ [CONFIG_PATH]: '{"mode":"shadow"}' })); + expect(config.configVersion).toBe(1); + }); + + it("accepts version 1 explicitly", () => { + const config = loadJudgeConfig( + deps({ [CONFIG_PATH]: '{"version":1,"mode":"shadow"}' }), + ); + expect(config.configVersion).toBe(1); + expect(config.diagnostics).toEqual([]); + }); + + it("accepts version 2", () => { + const config = loadJudgeConfig( + deps({ [CONFIG_PATH]: '{"version":2,"mode":"shadow"}' }), + ); + expect(config.configVersion).toBe(2); + expect(config.diagnostics).toEqual([]); + }); + + it("rejects an unknown version to all defaults with a diagnostic", () => { + const config = loadJudgeConfig( + deps({ [CONFIG_PATH]: '{"version":3,"mode":"enforce"}' }), + ); + expect(config).toMatchObject({ configVersion: 1, mode: "shadow" }); + expect(config.diagnostics[0]?.key).toBe("version"); + }); +}); + +describe("loadJudgeConfig — v1 enforce migration (fail-closed)", () => { + it("downgrades v1 enforce to shadow with a migration diagnostic", () => { + for (const raw of ['{"mode":"enforce"}', '{"version":1,"mode":"enforce"}']) { + const config = loadJudgeConfig(deps({ [CONFIG_PATH]: raw })); + expect(config.mode).toBe("shadow"); + expect(config.diagnostics).toEqual([ + { + key: "mode", + problem: expect.stringMatching(/"version": 2/), + fallback: "shadow (v1 enforce requires explicit migration)", + }, + ]); + } + }); + + it("keeps v1 shadow without diagnostics", () => { + const config = loadJudgeConfig(deps({ [CONFIG_PATH]: '{"mode":"shadow"}' })); + expect(config.mode).toBe("shadow"); + expect(config.diagnostics).toEqual([]); + }); +}); + describe("loadJudgeConfig — mode", () => { - it("accepts shadow and enforce", () => { + it("accepts shadow and enforce in v2", () => { expect( - loadJudgeConfig(deps({ [CONFIG_PATH]: '{"mode":"shadow"}' })).mode, + loadJudgeConfig( + deps({ [CONFIG_PATH]: '{"version":2,"mode":"shadow"}' }), + ).mode, ).toBe("shadow"); expect( - loadJudgeConfig(deps({ [CONFIG_PATH]: '{"mode":"enforce"}' })).mode, + loadJudgeConfig( + deps({ [CONFIG_PATH]: '{"version":2,"mode":"enforce"}' }), + ).mode, ).toBe("enforce"); }); it("resolves an unknown mode to shadow with a diagnostic", () => { const config = loadJudgeConfig( - deps({ [CONFIG_PATH]: '{"mode":"yolo"}' }), + deps({ [CONFIG_PATH]: '{"version":2,"mode":"yolo"}' }), ); expect(config.mode).toBe("shadow"); expect(config.diagnostics).toEqual([ @@ -70,18 +129,72 @@ describe("loadJudgeConfig — mode", () => { }); it("resolves a missing mode to shadow without diagnostics", () => { - const config = loadJudgeConfig(deps({ [CONFIG_PATH]: "{}" })); + const config = loadJudgeConfig(deps({ [CONFIG_PATH]: '{"version":2}' })); expect(config.mode).toBe("shadow"); expect(config.diagnostics).toEqual([]); }); }); +describe("loadJudgeConfig — v2 judge model", () => { + it("parses an explicit fixed judge model", () => { + const config = loadJudgeConfig( + deps({ + [CONFIG_PATH]: + '{"version":2,"mode":"enforce","model":{"provider":"openai-codex","id":"gpt-5.6-sol"}}', + }), + ); + expect(config.mode).toBe("enforce"); + expect(config.judgeModel).toEqual({ + provider: "openai-codex", + id: "gpt-5.6-sol", + }); + expect(config.diagnostics).toEqual([]); + }); + + it("allows a judge model in shadow mode too", () => { + const config = loadJudgeConfig( + deps({ + [CONFIG_PATH]: + '{"version":2,"mode":"shadow","model":{"provider":"p","id":"m"}}', + }), + ); + expect(config.mode).toBe("shadow"); + expect(config.judgeModel).toEqual({ provider: "p", id: "m" }); + }); + + it.each([ + '{"version":2,"model":"openai-codex"}', + '{"version":2,"model":{"provider":"p"}}', + '{"version":2,"model":{"id":"m"}}', + '{"version":2,"model":{"provider":"","id":"m"}}', + '{"version":2,"model":{"provider":"p","id":""}}', + '{"version":2,"model":{"provider":1,"id":"m"}}', + '{"version":2,"model":[]}', + ])("fails a malformed model %j closed to shadow with a diagnostic", (raw) => { + const config = loadJudgeConfig(deps({ [CONFIG_PATH]: raw })); + expect(config.mode).toBe("shadow"); + expect(config.judgeModel).toBeUndefined(); + expect(config.diagnostics[0]?.key).toBe("model"); + }); + + it("ignores a model field in v1 with a diagnostic", () => { + const config = loadJudgeConfig( + deps({ [CONFIG_PATH]: '{"model":{"provider":"p","id":"m"}}' }), + ); + expect(config.configVersion).toBe(1); + expect(config.judgeModel).toBeUndefined(); + expect(config.diagnostics[0]?.key).toBe("model"); + }); +}); + describe("loadJudgeConfig — timeout boundaries", () => { it.each([4_999, 30_001, 0, -5_000, 15.5, NaN, Infinity, "20000"])( "rejects invalid timeoutMs %p with fallback to 15,000", (value) => { const config = loadJudgeConfig( - deps({ [CONFIG_PATH]: JSON.stringify({ timeoutMs: value }) }), + deps({ + [CONFIG_PATH]: JSON.stringify({ version: 2, timeoutMs: value }), + }), ); expect(config.timeoutMs).toBe(15_000); expect(config.timeoutCohort).toBe("default"); @@ -91,16 +204,20 @@ describe("loadJudgeConfig — timeout boundaries", () => { it("accepts the inclusive boundaries 5,000 and 30,000", () => { expect( - loadJudgeConfig(deps({ [CONFIG_PATH]: '{"timeoutMs":5000}' })), + loadJudgeConfig( + deps({ [CONFIG_PATH]: '{"version":2,"timeoutMs":5000}' }), + ), ).toMatchObject({ timeoutMs: 5_000, timeoutCohort: 5_000 }); expect( - loadJudgeConfig(deps({ [CONFIG_PATH]: '{"timeoutMs":30000}' })), + loadJudgeConfig( + deps({ [CONFIG_PATH]: '{"version":2,"timeoutMs":30000}' }), + ), ).toMatchObject({ timeoutMs: 30_000, timeoutCohort: 30_000 }); }); it("marks an explicit default timeout as the default cohort", () => { const config = loadJudgeConfig( - deps({ [CONFIG_PATH]: '{"timeoutMs":15000}' }), + deps({ [CONFIG_PATH]: '{"version":2,"timeoutMs":15000}' }), ); expect(config.timeoutCohort).toBe("default"); expect(config.diagnostics).toEqual([]); @@ -108,7 +225,7 @@ describe("loadJudgeConfig — timeout boundaries", () => { it("marks a non-default timeout as a distinct cohort", () => { const config = loadJudgeConfig( - deps({ [CONFIG_PATH]: '{"timeoutMs":30000}' }), + deps({ [CONFIG_PATH]: '{"version":2,"timeoutMs":30000}' }), ); expect(config.timeoutCohort).toBe(30_000); }); @@ -117,13 +234,18 @@ describe("loadJudgeConfig — timeout boundaries", () => { describe("loadJudgeConfig — snapshot immutability", () => { it("returns an immutable effective-config snapshot", () => { const config = loadJudgeConfig( - deps({ [CONFIG_PATH]: '{"mode":"enforce","timeoutMs":20000}' }), + deps({ + [CONFIG_PATH]: + '{"version":2,"mode":"enforce","timeoutMs":20000,"model":{"provider":"p","id":"m"}}', + }), ); expect(Object.isFrozen(config)).toBe(true); expect(config).toEqual({ + configVersion: 2, mode: "enforce", timeoutMs: 20_000, timeoutCohort: 20_000, + judgeModel: { provider: "p", id: "m" }, diagnostics: [], }); }); diff --git a/packages/pi-permission-ai-judge/test/highrisk.test.ts b/packages/pi-permission-ai-judge/test/highrisk.test.ts new file mode 100644 index 0000000..fd1a400 --- /dev/null +++ b/packages/pi-permission-ai-judge/test/highrisk.test.ts @@ -0,0 +1,208 @@ +import { describe, expect, it } from "vitest"; +import { classifyHighRisk } from "../src/highrisk"; + +describe("classifyHighRisk — data_loss", () => { + it.each([ + "git clean -xfd", + "git clean -fdx", + "git clean -fxd", + "git clean -f -x -d", + "git clean --force -xd", + "git reset --hard", + "git reset --hard HEAD~3", + "git reset --hard origin/main", + "git checkout -- .", + "git checkout .", + "git restore .", + "git restore -- .", + "rm -rf ~", + "rm -rf /", + "rm -rf ~/*", + "rm -rf $HOME", + "rm -rf $HOME/projects", + "rm -fr ~", + ])("flags %j", (command) => { + expect(classifyHighRisk(command)?.category).toBe("data_loss"); + }); + + it.each([ + "git clean -nxd", + "git clean -nd", + "git reset --soft HEAD~1", + "git checkout main", + "git checkout -- file.txt", + "git restore file.txt", + "rm -rf build/", + "rm -rf ./dist", + "rm -r build", + "rm file.txt", + "echo git clean -xfd", + "git status", + ])("does not flag %j", (command) => { + expect(classifyHighRisk(command)).toBeUndefined(); + }); +}); + +describe("classifyHighRisk — history_rewrite", () => { + it.each([ + "git push --force", + "git push -f", + "git push --force origin main", + "git push --force-with-lease origin main", + "git push origin +main", + ])("flags %j", (command) => { + expect(classifyHighRisk(command)?.category).toBe("history_rewrite"); + }); + + it.each(["git push origin main", "git push", "git push --tags"])( + "does not flag %j", + (command) => { + expect(classifyHighRisk(command)).toBeUndefined(); + }, + ); +}); + +describe("classifyHighRisk — publish_deploy", () => { + it.each([ + "npm publish", + "npm publish --access public", + "pnpm publish", + "yarn publish", + "cargo publish", + "terraform destroy", + "terraform destroy -auto-approve", + ])("flags %j", (command) => { + expect(classifyHighRisk(command)?.category).toBe("publish_deploy"); + }); + + it.each(["npm run publish", "npm install", "pnpm test", "terraform plan", "terraform apply"])( + "does not flag %j", + (command) => { + expect(classifyHighRisk(command)).toBeUndefined(); + }, + ); +}); + +describe("classifyHighRisk — system_modify", () => { + it.each([ + "sudo rm /etc/hosts", + "sudo -i", + "sudo apt-get install ripgrep", + "mkfs.ext4 /dev/sda1", + "mkfs /dev/sdb", + "dd if=/dev/zero of=/dev/sda bs=1M", + "shutdown now", + "shutdown -h now", + "reboot", + "halt", + "poweroff", + ])("flags %j", (command) => { + expect(classifyHighRisk(command)?.category).toBe("system_modify"); + }); + + it.each([ + "dd if=/dev/zero of=/tmp/img bs=1M count=10", + "echo sudo", + "cat /etc/hosts", + ])("does not flag %j", (command) => { + expect(classifyHighRisk(command)).toBeUndefined(); + }); +}); + +describe("classifyHighRisk — credential_access", () => { + it.each([ + "cat ~/.ssh/id_rsa", + "cat ~/.ssh/id_ed25519", + "less ~/.ssh/id_ed25519", + "cat ~/.aws/credentials", + "head ~/.netrc", + "cat ~/.config/gcloud/application_default_credentials.json", + "rm ~/.ssh/id_ed25519", + "mv ~/.ssh/authorized_keys /tmp", + "cp ~/.gnupg/secring.gpg .", + "cat $HOME/.ssh/id_rsa", + "cat $HOME/.aws/credentials", + ])("flags %j", (command) => { + expect(classifyHighRisk(command)?.category).toBe("credential_access"); + }); + + it.each([ + "echo x > ~/.aws/credentials", + "echo x >> ~/.ssh/authorized_keys", + "ssh-keygen foo 2> ~/.aws/credentials", + "printf '%s' key > ~/.ssh/id_ed25519", + "curl -s url | tee ~/.netrc", + ])("flags credential replacement %j", (command) => { + expect(classifyHighRisk(command)).toMatchObject({ + category: "credential_access", + }); + }); + + it.each([ + "cat ~/.ssh/config", + "cat ~/.ssh/known_hosts", + "cat package.json", + "ls ~/.ssh", + "cat ~/.bashrc", + "cat id_rsa", + "echo x > ~/.bashrc", + "curl -s url | tee notes.txt", + ])("does not flag %j", (command) => { + expect(classifyHighRisk(command)).toBeUndefined(); + }); +}); + +describe("classifyHighRisk — compound inputs", () => { + it("flags when any unit matches", () => { + expect(classifyHighRisk("pnpm test && git clean -xfd")?.category).toBe( + "data_loss", + ); + expect(classifyHighRisk("git push --force; echo done")?.category).toBe( + "history_rewrite", + ); + expect(classifyHighRisk("npm publish | tee log")?.category).toBe( + "publish_deploy", + ); + }); + + it("flags single-`&` background compound units", () => { + expect(classifyHighRisk("echo ready & npm publish")?.category).toBe( + "publish_deploy", + ); + expect(classifyHighRisk("sleep 5 & rm -rf ~")?.category).toBe( + "data_loss", + ); + }); + + it("does not flag separators inside quotes or escapes", () => { + expect(classifyHighRisk("printf 'x; npm publish;'")).toBeUndefined(); + expect(classifyHighRisk("echo \"git push --force\"")).toBeUndefined(); + expect(classifyHighRisk("echo git\\;npm\\;publish")).toBeUndefined(); + expect(classifyHighRisk("echo 'rm -rf ~'")).toBeUndefined(); + }); + + it("treats a quoted command argument as one token, not a command", () => { + // git commit -m "junk; sudo rm /" quotes an argument, not a unit. + expect(classifyHighRisk("git commit -m 'do not npm publish'")).toBeUndefined(); + }); + + it("flags high-risk units even when another unit quotes text", () => { + expect(classifyHighRisk("echo 'a;b' && git push --force")).toMatchObject({ + category: "history_rewrite", + }); + }); + + it("returns undefined for all-benign compounds", () => { + expect(classifyHighRisk("pnpm test && pnpm check")).toBeUndefined(); + expect(classifyHighRisk("echo a; echo b | wc -l")).toBeUndefined(); + expect(classifyHighRisk("sleep 5 & echo done")).toBeUndefined(); + }); + + it("reports the matching rule for audit observability", () => { + const match = classifyHighRisk("git clean -xfd"); + expect(match).toMatchObject({ + category: "data_loss", + rule: expect.stringMatching(/^git_clean/), + }); + }); +}); diff --git a/packages/pi-permission-ai-judge/test/judge.test.ts b/packages/pi-permission-ai-judge/test/judge.test.ts index cae7da3..0940ac2 100644 --- a/packages/pi-permission-ai-judge/test/judge.test.ts +++ b/packages/pi-permission-ai-judge/test/judge.test.ts @@ -3,18 +3,10 @@ import { evaluateEnforceAuthority, type EnforceGateState, } from "../src/judge"; -import { - loadPromotionRecords, - resolvePromotionGates, - type CandidateIdentity, -} from "../src/promotion"; const ALL_OPEN: EnforceGateState = { auditHealthy: true, telemetryHealth: "healthy", - cohortQualified: true, - ownerApprovalRecorded: true, - activationRecorded: true, resultKind: "judgment", verdict: "allow", reviewAcknowledged: true, @@ -23,7 +15,10 @@ const ALL_OPEN: EnforceGateState = { }; describe("evaluateEnforceAuthority — every gate independently forces defer", () => { - it("allows only when every gate holds", () => { + it("allows when every gate holds — no promotion records required", () => { + // ADR 0008: the promotion gates (cohort qualification, owner + // approval, activation) are no longer authority inputs; a session + // with zero promotion records can hold Enforce authority. expect(evaluateEnforceAuthority(ALL_OPEN)).toEqual({ kind: "allow" }); }); @@ -37,9 +32,6 @@ describe("evaluateEnforceAuthority — every gate independently forces defer", ( { name: "telemetry disabled", patch: { telemetryHealth: "disabled" }, expectedReason: "telemetry_disabled" }, { name: "telemetry write failed", patch: { telemetryHealth: "write_failed" }, expectedReason: "telemetry_write_failed" }, { name: "telemetry integrity anomaly", patch: { telemetryHealth: "integrity_anomaly" }, expectedReason: "telemetry_integrity_anomaly" }, - { name: "cohort not qualified", patch: { cohortQualified: false }, expectedReason: "cohort_not_qualified" }, - { name: "owner approval absent", patch: { ownerApprovalRecorded: false }, expectedReason: "owner_approval_absent" }, - { name: "activation absent", patch: { activationRecorded: false }, expectedReason: "activation_absent" }, { name: "preflight result", patch: { resultKind: "preflight_defer" }, expectedReason: "result_preflight_defer" }, { name: "infrastructure result", patch: { resultKind: "infrastructure_failure" }, expectedReason: "result_infrastructure_failure" }, { name: "semantic deny verdict", patch: { verdict: "deny" }, expectedReason: "verdict_deny" }, @@ -58,86 +50,3 @@ describe("evaluateEnforceAuthority — every gate independently forces defer", ( }); } }); - -describe("evaluateEnforceAuthority — production state with no promotion records", () => { - // The real post-PIEXTENSIO-21 seam: an empty records file (the normal - // pre-promotion state) closes all promotion gates, so every mode and - // telemetry state defers — mechanically identical to v0.1's hardcoded - // closure, now derived from actual storage. - const emptySnapshot = loadPromotionRecords({ - agentDir: "/nonexistent-agent-dir", - }); - const identity: CandidateIdentity = { - judge: "@sikongjueluo/pi-permission-ai-judge@0.0.1", - permissionSystem: "25.4.0", - provider: "openai-codex", - model: "gpt-5.6-sol", - api: "openai-codex-responses", - promptVersion: "bash-shadow-v4", - toolSchemaVersion: "report-verdict-v1", - reviewSchemaVersion: "1", - timeoutCohort: 30000, - }; - - it("never grants authority for any mode or telemetry state without records", () => { - const modes = ["shadow", "enforce"] as const; - const healths = ["healthy", "disabled", "write_failed", "integrity_anomaly"] as const; - for (const mode of modes) { - for (const health of healths) { - const gates = resolvePromotionGates(emptySnapshot, identity); - const outcome = evaluateEnforceAuthority({ - ...ALL_OPEN, - mode, - telemetryHealth: health, - ...gates, - }); - expect(outcome.kind).toBe("defer"); - } - } - }); - - it("blocks enforce on the cohort gate first even with healthy audit", () => { - const gates = resolvePromotionGates(emptySnapshot, identity); - const outcome = evaluateEnforceAuthority({ - ...ALL_OPEN, - ...gates, - }); - expect(outcome).toEqual({ - kind: "defer", - blockedBy: "cohort_not_qualified", - }); - }); - - it("grants authority only when every record kind exists for the exact identity", () => { - const records = [ - { - kind: "cohort_qualified", - candidateIdentity: identity, - recordedAt: "2026-08-20T12:00:00Z", - basis: "cohort test", - }, - { - kind: "owner_approval", - candidateIdentity: identity, - recordedAt: "2026-08-20T12:01:00Z", - basis: "approved", - }, - { - kind: "activation", - candidateIdentity: identity, - recordedAt: "2026-08-20T12:02:00Z", - basis: "activated", - }, - ] as const; - const snapshot = { - records, - healthy: true, - diagnostic: null, - path: "unused", - }; - const gates = resolvePromotionGates(snapshot, identity); - expect(evaluateEnforceAuthority({ ...ALL_OPEN, ...gates })).toEqual({ - kind: "allow", - }); - }); -}); diff --git a/packages/pi-permission-ai-judge/test/lifecycle.test.ts b/packages/pi-permission-ai-judge/test/lifecycle.test.ts index 3d9cee1..cddd836 100644 --- a/packages/pi-permission-ai-judge/test/lifecycle.test.ts +++ b/packages/pi-permission-ai-judge/test/lifecycle.test.ts @@ -50,7 +50,6 @@ import { unpublishPermissionsService, } from "@gotgenes/pi-permission-system"; import extension from "../src/index"; -import { appendPromotionRecord, type CandidateIdentity } from "../src/promotion"; import { PROMPT_VERSION, TOOL_SCHEMA_VERSION } from "../src/prompt"; function createFakePi(): { @@ -542,33 +541,39 @@ describe("AI judge lifecycle", () => { }); }); -describe("AI judge Enforce authority seam (PIEXTENSIO-21)", () => { +describe("AI judge Enforce authority seam (PIEXTENSIO-23, ADR 0008)", () => { beforeEach(() => { createMockAgentDir(); - writeFileSync( - join(mockAgentDir.dir, "pi-permission-ai-judge.config.json"), - JSON.stringify({ mode: "enforce" }), - ); }); - /** Records identity matching the lifecycle fake model + static fields. */ - function lifecycleIdentity(): CandidateIdentity { - return { - judge: "@sikongjueluo/pi-permission-ai-judge@0.0.1", - permissionSystem: "25.4.0", - provider: "test-provider", - model: "test-model", - api: "openai-codex-responses", - promptVersion: PROMPT_VERSION, - toolSchemaVersion: TOOL_SCHEMA_VERSION, - reviewSchemaVersion: "1", - timeoutCohort: "default", - }; + function writeConfig(config: Record): void { + writeFileSync( + join(mockAgentDir.dir, "pi-permission-ai-judge.config.json"), + JSON.stringify(config), + ); } - async function runAsk(): Promise<{ + interface RunAskOptions { + /** Config file contents (written before session start). */ + config: Record; + /** Command unit for the ask; full command becomes `pnpm test && `. */ + command?: string; + /** modelRegistry.find result for a configured judge model (when set). */ + findResult?: Model | undefined; + /** Whether the found model has configured auth. */ + authConfigured?: boolean; + /** Overridable model verdict. */ + response?: AssistantMessage; + } + + async function runAsk( + options: RunAskOptions, + ): Promise<{ verdict: { kind: string }; reviews: Array<{ event: string; details?: Record }>; + notify: ReturnType; + complete: ReturnType; + find: ReturnType; }> { let authorize: Authorizer["authorize"] | undefined; const service = { @@ -582,28 +587,45 @@ describe("AI judge Enforce authority seam (PIEXTENSIO-21)", () => { publishPermissionsService(service); publishedService = service; - const complete = vi.fn(async () => modelResponse()); + const complete = vi.fn(async () => options.response ?? modelResponse()); + const find = vi.fn(() => options.findResult); + const hasConfiguredAuth = vi.fn(() => options.authConfigured !== false); + const notify = vi.fn(); + const sessionManager = fakeSessionManager(); const ctx = { hasUI: true, - sessionManager: fakeSessionManager(), + sessionManager, model: { - id: "test-model", - provider: "test-provider", + id: "session-model", + provider: "session-provider", api: "openai-codex-responses", } as Model, - modelRegistry: { complete }, - ui: { notify: vi.fn() }, + modelRegistry: { complete, find, hasConfiguredAuth }, + ui: { notify }, } as unknown as ExtensionContext; const harness = createFakePi(); extension(harness.pi); + writeConfig(options.config); harness.start(ctx); harness.ready(); expect(authorize).toBeDefined(); + const unit = options.command ?? "git push origin main"; + const askDetails = ask(); + askDetails.command = unit; + const request = askDetails.payload.request as { value: string }; + request.value = unit; + const evidenceEntry = ( + askDetails.payload as { evidence: ReadonlyArray<{ label: string; text: string }> } + ).evidence[0]; + if (evidenceEntry !== undefined) { + evidenceEntry.text = `pnpm test && ${unit}`; + } + const reviews: Array<{ event: string; details?: Record }> = []; const verdict = await authorize!( - ask(), + askDetails, { checkPermission: vi.fn(), getToolPermission: vi.fn(), @@ -614,45 +636,13 @@ describe("AI judge Enforce authority seam (PIEXTENSIO-21)", () => { }, ); harness.shutdown(); - return { verdict, reviews }; + return { verdict, reviews, notify, complete, find }; } - it("defers in enforce mode when no promotion records exist", async () => { - const { verdict, reviews } = await runAsk(); - expect(verdict).toEqual({ kind: "defer" }); - expect(reviews).toMatchObject([ - { - event: "ai_bash_judge.result", - details: expect.objectContaining({ - mode: "enforce", - verdict: "allow", - effectiveVerdict: "defer", - authorityBlockedBy: "cohort_not_qualified", - }), - }, - ]); - }); - - it("grants authority in enforce mode only with all three exact-identity records", async () => { - const identity = lifecycleIdentity(); - for (const [kind, basis] of [ - ["cohort_qualified", "cohort piextensio-test"], - ["owner_approval", "approved for test"], - ["activation", "activated for test"], - ] as const) { - expect( - appendPromotionRecord({ - agentDir: mockAgentDir.dir, - record: { - kind, - candidateIdentity: identity, - recordedAt: "2026-08-21T10:00:00Z", - basis, - }, - }), - ).toBeNull(); - } - const { verdict, reviews } = await runAsk(); + it("grants authority in v2 enforce mode with no promotion records", async () => { + const { verdict, reviews } = await runAsk({ + config: { version: 2, mode: "enforce" }, + }); expect(verdict).toEqual({ kind: "allow" }); expect(reviews).toMatchObject([ { @@ -662,73 +652,227 @@ describe("AI judge Enforce authority seam (PIEXTENSIO-21)", () => { verdict: "allow", effectiveVerdict: "allow", authorityBlockedBy: null, + modelSource: "session", }), }, ]); }); - it("defers in enforce mode when records exist for another identity", async () => { - const identity = { ...lifecycleIdentity(), model: "other-model" }; - for (const kind of [ - "cohort_qualified", - "owner_approval", - "activation", - ] as const) { - appendPromotionRecord({ - agentDir: mockAgentDir.dir, - record: { - kind, - candidateIdentity: identity, - recordedAt: "2026-08-21T10:00:00Z", - basis: "other identity", - }, - }); - } - const { verdict, reviews } = await runAsk(); - expect(verdict).toEqual({ kind: "defer" }); - expect(reviews).toMatchObject([ - { - event: "ai_bash_judge.result", - details: expect.objectContaining({ - effectiveVerdict: "defer", - authorityBlockedBy: "cohort_not_qualified", - }), - }, - ]); - }); - - it("never grants authority in shadow mode regardless of records", async () => { - writeFileSync( - join(mockAgentDir.dir, "pi-permission-ai-judge.config.json"), - JSON.stringify({ mode: "shadow" }), - ); - const identity = lifecycleIdentity(); - for (const kind of [ - "cohort_qualified", - "owner_approval", - "activation", - ] as const) { - appendPromotionRecord({ - agentDir: mockAgentDir.dir, - record: { - kind, - candidateIdentity: identity, - recordedAt: "2026-08-21T10:00:00Z", - basis: "shadow still defers", - }, - }); - } - const { verdict, reviews } = await runAsk(); + it("downgrades v1 enforce to shadow with a migration diagnostic notification", async () => { + const { verdict, reviews, notify, complete } = await runAsk({ + config: { mode: "enforce" }, + }); expect(verdict).toEqual({ kind: "defer" }); expect(reviews).toMatchObject([ { event: "ai_bash_judge.result", details: expect.objectContaining({ mode: "shadow", + verdict: "allow", effectiveVerdict: "defer", authorityBlockedBy: "mode_shadow", }), }, ]); + const notified = notify.mock.calls.map((call) => String(call[0])); + expect( + notified.some((message) => + /v1 enforce requires explicit migration|version 2/.test(message), + ), + ).toBe(true); + expect(complete).toHaveBeenCalledTimes(1); + }); + + it("defers immediately on a high-risk command in enforce mode without calling the model", async () => { + const { verdict, reviews, complete } = await runAsk({ + config: { version: 2, mode: "enforce" }, + command: "git clean -xfd", + }); + expect(verdict).toEqual({ kind: "defer" }); + expect(complete).not.toHaveBeenCalled(); + expect(reviews).toMatchObject([ + { + event: "ai_bash_judge.result", + details: expect.objectContaining({ + resultKind: "preflight_defer", + verdict: null, + effectiveVerdict: "defer", + modelCalled: false, + code: "high_risk_override", + riskCategory: "data_loss", + riskRule: expect.any(String), + }), + }, + ]); + }); + + it("still calls the model on a high-risk command in shadow mode, records the override, and defers", async () => { + const { verdict, reviews, complete } = await runAsk({ + config: { version: 2, mode: "shadow" }, + command: "git clean -xfd", + }); + expect(complete).toHaveBeenCalledTimes(1); + expect(verdict).toEqual({ kind: "defer" }); + expect(reviews).toMatchObject([ + { + event: "ai_bash_judge.result", + details: expect.objectContaining({ + resultKind: "judgment", + verdict: "allow", + effectiveVerdict: "defer", + riskOverride: { category: "data_loss", rule: expect.any(String) }, + }), + }, + ]); + }); + + it("resolves a configured v2 judge model instead of the session model", async () => { + const { verdict, reviews, complete, find } = await runAsk({ + config: { + version: 2, + mode: "enforce", + model: { provider: "fixed-provider", id: "fixed-model" }, + }, + findResult: { + id: "fixed-model", + provider: "fixed-provider", + api: "openai-codex-responses", + } as Model, + }); + expect(find).toHaveBeenCalledWith("fixed-provider", "fixed-model"); + expect(complete).toHaveBeenCalledTimes(1); + expect((complete.mock.calls[0] as unknown[])[0]).toMatchObject({ + id: "fixed-model", + provider: "fixed-provider", + }); + expect(verdict).toEqual({ kind: "allow" }); + expect(reviews[0]?.details).toMatchObject({ + modelSource: "configured", + provider: "fixed-provider", + model: "fixed-model", + }); + }); + + it("defers with judge_model_unavailable when a configured model cannot be resolved, without fallback", async () => { + const { verdict, reviews, complete } = await runAsk({ + config: { + version: 2, + mode: "enforce", + model: { provider: "ghost-provider", id: "ghost-model" }, + }, + findResult: undefined, + }); + expect(verdict).toEqual({ kind: "defer" }); + expect(complete).not.toHaveBeenCalled(); + expect(reviews).toMatchObject([ + { + event: "ai_bash_judge.result", + details: expect.objectContaining({ + resultKind: "infrastructure_failure", + code: "judge_model_unavailable", + modelCalled: false, + provider: "ghost-provider", + model: "ghost-model", + }), + }, + ]); + }); + + it("keeps riskOverride on the early judge_model_unavailable row for a high-risk shadow ask", async () => { + const { reviews } = await runAsk({ + config: { + version: 2, + mode: "shadow", + model: { provider: "ghost-provider", id: "ghost-model" }, + }, + command: "git clean -xfd", + findResult: undefined, + }); + expect(reviews[0]?.details).toMatchObject({ + code: "judge_model_unavailable", + riskOverride: { category: "data_loss", rule: expect.any(String) }, + }); + }); + + it("defers with judge_model_unavailable when a configured model lacks auth", async () => { + const { verdict, reviews, complete } = await runAsk({ + config: { + version: 2, + mode: "enforce", + model: { provider: "p", id: "m" }, + }, + findResult: { id: "m", provider: "p", api: "openai-codex-responses" } as Model, + authConfigured: false, + }); + expect(verdict).toEqual({ kind: "defer" }); + expect(complete).not.toHaveBeenCalled(); + expect(reviews[0]?.details).toMatchObject({ + resultKind: "infrastructure_failure", + code: "judge_model_unavailable", + }); + }); + + it("notifies once per session in v2 enforce mode with the judge model and risk contract", async () => { + let authorize: Authorizer["authorize"] | undefined; + const service = { + registerAuthorizer: vi.fn((_name, callback) => { + authorize = callback; + return vi.fn(); + }), + checkPermission: vi.fn(), + getToolPermission: vi.fn(), + } as unknown as PermissionsService; + publishPermissionsService(service); + publishedService = service; + + const complete = vi.fn(async () => modelResponse()); + const notify = vi.fn(); + const ctx = { + hasUI: true, + sessionManager: fakeSessionManager(), + model: { + id: "session-model", + provider: "session-provider", + api: "openai-codex-responses", + } as Model, + modelRegistry: { complete, find: vi.fn(), hasConfiguredAuth: vi.fn(() => true) }, + ui: { notify }, + } as unknown as ExtensionContext; + + const harness = createFakePi(); + extension(harness.pi); + writeConfig({ version: 2, mode: "enforce" }); + harness.start(ctx); + harness.ready(); + + const enforceNotice = notify.mock.calls.filter((call) => + /Enforce/i.test(String(call[0])), + ); + expect(enforceNotice).toHaveLength(1); + expect(String(enforceNotice[0]?.[0])).toContain( + "session-provider/session-model", + ); + expect(String(enforceNotice[0]?.[0])).toMatch(/risk/i); + + // Two more asks must not repeat the notification. + for (let i = 0; i < 2; i += 1) { + await authorize!(ask(), { checkPermission: vi.fn(), getToolPermission: vi.fn() }, { + review: vi.fn(), + debug: vi.fn(), + }); + } + expect( + notify.mock.calls.filter((call) => /Enforce/i.test(String(call[0]))), + ).toHaveLength(1); + harness.shutdown(); + }); + + it("does not show the enforce notification in shadow mode", async () => { + const { notify } = await runAsk({ + config: { version: 2, mode: "shadow" }, + }); + expect( + notify.mock.calls.filter((call) => /Enforce/i.test(String(call[0]))), + ).toHaveLength(0); }); }); diff --git a/packages/pi-permission-ai-judge/test/promotion.test.ts b/packages/pi-permission-ai-judge/test/promotion.test.ts deleted file mode 100644 index 09ba23d..0000000 --- a/packages/pi-permission-ai-judge/test/promotion.test.ts +++ /dev/null @@ -1,260 +0,0 @@ -import { mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { dirname, join } from "node:path"; -import { afterEach, beforeEach, describe, expect, it } from "vitest"; -import { - appendPromotionRecord, - loadPromotionRecords, - promotionRecordsPath, - resolvePromotionGates, - type CandidateIdentity, - type PromotionRecord, -} from "../src/promotion"; - -const IDENTITY: CandidateIdentity = { - judge: "@sikongjueluo/pi-permission-ai-judge@0.0.1", - permissionSystem: "25.4.0", - provider: "openai-codex", - model: "gpt-5.6-sol", - api: "openai-codex-responses", - promptVersion: "bash-shadow-v4", - toolSchemaVersion: "report-verdict-v1", - reviewSchemaVersion: "1", - timeoutCohort: 30000, -}; - -function record( - kind: PromotionRecord["kind"], - identity: CandidateIdentity = IDENTITY, - basis = "test basis", -): PromotionRecord { - return { - kind, - candidateIdentity: identity, - recordedAt: "2026-08-20T12:00:00Z", - basis, - }; -} - -function line(value: unknown): string { - return `${JSON.stringify(value)}\n`; -} - -/** writeFileSync, but creating the records directory first. */ -function writeRecords(dir: string, content: string): void { - const path = promotionRecordsPath(dir); - mkdirSync(dirname(path), { recursive: true }); - writeFileSync(path, content); -} - -describe("promotion records — loadPromotionRecords", () => { - let dir: string; - beforeEach(() => { - dir = mkdtempSync(join(tmpdir(), "ai-judge-promotion-")); - }); - afterEach(() => { - rmSync(dir, { recursive: true, force: true }); - }); - - it("treats a missing file as the healthy pre-promotion state", () => { - const snapshot = loadPromotionRecords({ agentDir: dir }); - expect(snapshot).toEqual({ - records: [], - healthy: true, - diagnostic: null, - path: promotionRecordsPath(dir), - }); - }); - - it("parses well-formed records of every kind", () => { - writeRecords( - dir, - [record("cohort_qualified"), record("owner_approval"), record("activation")] - .map(line) - .join(""), - ); - const snapshot = loadPromotionRecords({ agentDir: dir }); - expect(snapshot.healthy).toBe(true); - expect(snapshot.records).toHaveLength(3); - expect(snapshot.records.map((r) => r.kind)).toEqual([ - "cohort_qualified", - "owner_approval", - "activation", - ]); - }); - - it("fails closed on malformed JSON lines", () => { - writeRecords( - dir, - line(record("cohort_qualified")) + "{not json\n", - ); - const snapshot = loadPromotionRecords({ agentDir: dir }); - expect(snapshot.healthy).toBe(false); - expect(snapshot.records).toEqual([]); - expect(snapshot.diagnostic).toContain("malformed"); - }); - - it("fails closed on shape-invalid records", () => { - writeRecords( - dir, - line({ kind: "activation" }), // missing identity/basis/recordedAt - ); - const snapshot = loadPromotionRecords({ agentDir: dir }); - expect(snapshot.healthy).toBe(false); - expect(snapshot.diagnostic).toContain("malformed"); - }); - - it("skips blank lines without failing", () => { - writeRecords( - dir, - "\n" + line(record("activation")) + "\n\n", - ); - const snapshot = loadPromotionRecords({ agentDir: dir }); - expect(snapshot.healthy).toBe(true); - expect(snapshot.records).toHaveLength(1); - }); -}); - -describe("promotion records — resolvePromotionGates", () => { - let dir: string; - beforeEach(() => { - dir = mkdtempSync(join(tmpdir(), "ai-judge-promotion-")); - }); - afterEach(() => { - rmSync(dir, { recursive: true, force: true }); - }); - - it("flips only its own gate per record kind", () => { - writeRecords(dir, line(record("owner_approval"))); - const snapshot = loadPromotionRecords({ agentDir: dir }); - expect(resolvePromotionGates(snapshot, IDENTITY)).toEqual({ - cohortQualified: false, - ownerApprovalRecorded: true, - activationRecorded: false, - }); - }); - - it("all three gates open with all three records for the exact identity", () => { - writeRecords( - dir, - [ - record("cohort_qualified", IDENTITY, "cohort id x"), - record("owner_approval", IDENTITY, "approved"), - record("activation", IDENTITY, "activated"), - ] - .map(line) - .join(""), - ); - const snapshot = loadPromotionRecords({ agentDir: dir }); - expect(resolvePromotionGates(snapshot, IDENTITY)).toEqual({ - cohortQualified: true, - ownerApprovalRecorded: true, - activationRecorded: true, - }); - }); - - const identityDrifts: ReadonlyArray<[string, Partial]> = [ - ["provider", { provider: "other-provider" }], - ["model", { model: "gpt-5.7" }], - ["api", { api: "openai-responses" }], - ["promptVersion", { promptVersion: "bash-shadow-v5" }], - ["toolSchemaVersion", { toolSchemaVersion: "report-verdict-v2" }], - ["reviewSchemaVersion", { reviewSchemaVersion: "2" }], - ["timeoutCohort", { timeoutCohort: "default" }], - ["permissionSystem", { permissionSystem: "25.5.0" }], - ["judge package", { judge: "@sikongjueluo/pi-permission-ai-judge@0.0.2" }], - ]; - for (const [name, patch] of identityDrifts) { - it(`keeps every gate closed when the live identity drifts on ${name}`, () => { - writeRecords( - dir, - [ - record("cohort_qualified"), - record("owner_approval"), - record("activation"), - ] - .map(line) - .join(""), - ); - const snapshot = loadPromotionRecords({ agentDir: dir }); - expect( - resolvePromotionGates(snapshot, { ...IDENTITY, ...patch }), - ).toEqual({ - cohortQualified: false, - ownerApprovalRecorded: false, - activationRecorded: false, - }); - }); - } - - it("closes every gate when the snapshot is unhealthy", () => { - writeRecords(dir, "garbage\n"); - const snapshot = loadPromotionRecords({ agentDir: dir }); - expect(snapshot.healthy).toBe(false); - expect(resolvePromotionGates(snapshot, IDENTITY)).toEqual({ - cohortQualified: false, - ownerApprovalRecorded: false, - activationRecorded: false, - }); - }); - - it("leaves other-identity records inert, not malformed", () => { - const other: CandidateIdentity = { - ...IDENTITY, - model: "glm-5.2", - }; - writeRecords( - dir, - [ - record("cohort_qualified", other), - record("activation", other), - ] - .map(line) - .join(""), - ); - const snapshot = loadPromotionRecords({ agentDir: dir }); - expect(snapshot.healthy).toBe(true); - expect(resolvePromotionGates(snapshot, IDENTITY)).toEqual({ - cohortQualified: false, - ownerApprovalRecorded: false, - activationRecorded: false, - }); - }); -}); - -describe("promotion records — appendPromotionRecord", () => { - let dir: string; - beforeEach(() => { - dir = mkdtempSync(join(tmpdir(), "ai-judge-promotion-")); - }); - afterEach(() => { - rmSync(dir, { recursive: true, force: true }); - }); - - it("appends a shape-valid record that round-trips through the loader", () => { - const error = appendPromotionRecord({ - agentDir: dir, - record: record("owner_approval", IDENTITY, "approved v4"), - now: () => "2026-08-21T09:00:00Z", - }); - expect(error).toBeNull(); - const raw = readFileSync(promotionRecordsPath(dir), "utf-8"); - expect(raw).toContain('"recordedAt":"2026-08-21T09:00:00Z"'); - expect(raw).toContain('"basis":"approved v4"'); - const snapshot = loadPromotionRecords({ agentDir: dir }); - expect(snapshot.healthy).toBe(true); - expect(resolvePromotionGates(snapshot, IDENTITY).ownerApprovalRecorded).toBe(true); - }); - - it("rejects a shape-invalid record without touching the file", () => { - const bad = { - kind: "activation", - candidateIdentity: { judge: "x" }, - recordedAt: "", - basis: "", - } as unknown as PromotionRecord; - const error = appendPromotionRecord({ agentDir: dir, record: bad }); - expect(error).toBe("record is not shape-valid"); - expect(loadPromotionRecords({ agentDir: dir }).records).toEqual([]); - }); -}); diff --git a/packages/pi-permission-ai-judge/tools/promotion-record.ts b/packages/pi-permission-ai-judge/tools/promotion-record.ts deleted file mode 100644 index d1522df..0000000 --- a/packages/pi-permission-ai-judge/tools/promotion-record.ts +++ /dev/null @@ -1,167 +0,0 @@ -/** - * PIEXTENSIO-21 owner action CLI (offline, package-local). - * - * Appends one promotion record to the Judge-owned records file - * (~/.pi/agent/extensions/pi-permission-ai-judge/promotion-records.jsonl). - * Never imported by online modules; no npm bin. The three record kinds map - * one-to-one to the Enforce truth-table promotion gates: - * - * cohort_qualified — after a declared replacement cohort meets the frozen - * floor, citing the cohort id + report reference - * owner_approval — the owner's explicit approval of that exact candidate - * activation — the distinct, explicit activation act - * - * Fail-closed contract: a record qualifies only its exact candidate - * identity; the judge re-derives identity from the live runtime, so a - * mismatched or malformed record is inert and malformed files close all - * gates. There is no CLI to *remove* authority records by design — - * rollback to Shadow is the config `mode` switch, and the append-only - * trail keeps the promotion history auditable. - * - * Usage: - * npx tsx tools/promotion-record.ts --kind cohort_qualified \ - * --provider openai-codex --model gpt-5.6-sol --api openai-codex-responses \ - * [--timeout-cohort 30000] --basis "cohort ; report docs/testing/..." - * - * judge/permission-system/prompt/tool-schema/review-schema versions are - * taken from the live package (src/prompt.ts + package constants) so the - * recorded identity cannot drift from the code that will check it. - */ - -import { getAgentDir } from "@earendil-works/pi-coding-agent"; -import { - appendPromotionRecord, - promotionRecordsPath, - type CandidateIdentity, - type PromotionRecordKind, -} from "../src/promotion"; -import { PROMPT_VERSION, TOOL_SCHEMA_VERSION } from "../src/prompt"; - -const JUDGE_IDENTITY = "@sikongjueluo/pi-permission-ai-judge@0.0.1"; -const PERMISSION_SYSTEM_VERSION = "25.4.0"; -const REVIEW_SCHEMA_VERSION = "1"; - -interface CliOptions { - kind: PromotionRecordKind; - provider: string; - model: string; - api: string; - timeoutCohort: "default" | number; - basis: string; -} - -function usage(): string { - return [ - "usage: promotion-record --kind ", - " --provider

--model --api ", - " [--timeout-cohort default|] --basis ", - ].join("\n"); -} - -function parseArgs(argv: readonly string[]): CliOptions | { error: string } { - const args = argv.slice(2); - const opts: Partial = {}; - for (let i = 0; i < args.length; i += 1) { - const arg = args[i] as string; - const value = args[i + 1]; - if ( - arg === "--kind" || - arg === "--provider" || - arg === "--model" || - arg === "--api" || - arg === "--basis" - ) { - if (value === undefined) return { error: `${arg} requires a value` }; - if (arg === "--kind") { - if ( - value !== "cohort_qualified" && - value !== "owner_approval" && - value !== "activation" - ) { - return { error: `unknown kind: ${value}` }; - } - opts.kind = value; - } - if (arg === "--provider") opts.provider = value; - if (arg === "--model") opts.model = value; - if (arg === "--api") opts.api = value; - if (arg === "--basis") opts.basis = value; - i += 1; - continue; - } - if (arg === "--timeout-cohort") { - if (value === undefined) return { error: `${arg} requires a value` }; - if (value === "default") { - opts.timeoutCohort = "default"; - } else { - const n = Number(value); - if (!Number.isInteger(n)) { - return { error: `--timeout-cohort must be default or an integer` }; - } - opts.timeoutCohort = n; - } - i += 1; - continue; - } - return { error: `unknown option: ${arg}\n${usage()}` }; - } - if ( - opts.kind === undefined || - opts.provider === undefined || - opts.model === undefined || - opts.api === undefined || - opts.basis === undefined || - opts.basis.length === 0 - ) { - return { error: usage() }; - } - return { - kind: opts.kind, - provider: opts.provider, - model: opts.model, - api: opts.api, - timeoutCohort: opts.timeoutCohort ?? "default", - basis: opts.basis, - }; -} - -function main(): number { - const parsed = parseArgs(process.argv); - if ("error" in parsed) { - process.stderr.write(`${parsed.error}\n`); - return 1; - } - const identity: CandidateIdentity = { - judge: JUDGE_IDENTITY, - permissionSystem: PERMISSION_SYSTEM_VERSION, - provider: parsed.provider, - model: parsed.model, - api: parsed.api, - promptVersion: PROMPT_VERSION, - toolSchemaVersion: TOOL_SCHEMA_VERSION, - reviewSchemaVersion: REVIEW_SCHEMA_VERSION, - timeoutCohort: parsed.timeoutCohort, - }; - const agentDir = getAgentDir(); - const error = appendPromotionRecord({ - agentDir, - record: { - kind: parsed.kind, - candidateIdentity: identity, - recordedAt: new Date().toISOString(), - basis: parsed.basis, - }, - }); - if (error !== null) { - process.stderr.write(`${error}\n`); - return 1; - } - process.stdout.write( - `appended ${parsed.kind} record to ${promotionRecordsPath(agentDir)}\n` + - `identity: ${JSON.stringify(identity)}\n` + - `note: records load at session start; restart sessions to pick this up\n`, - ); - return 0; -} - -process.exit(main());