Commit Graph
46 Commits
Author SHA1 Message Date
SikongJueluo 0903119a46 refactor(ai-judge): extract stage functions from top health hotspots
- split loadJudgeConfig into parseConfigVersion, parseMode, parseJudgeModelField, and parseTimeout with an explicit orchestration layer
- split analyzeShadowReviewLog into collectReviewEvents, joinTerminalRequest, and buildJoinedRow
- extract the authorizer callback from index.ts into module-level judgeAuthorize with shared preflight/infra result emitters
- extract the enforce-mode session notice into notifyEnforceActive
2026-08-22 01:24:50 +08:00
SikongJueluo 0737e01fca chore(fallow): adopt fallow and fix dead-code findings
- add fallow 3.17 as workspace devDependency with fallow/fallow:fix scripts
- generate .fallowrc.json modeling pi.extensions entries, diagnostic CLI, and tools as entry points
- remove export from 4 internal-only symbols
- drop unused pi-ai dependency from pi-permission-inner-cmd
2026-08-22 01:24:42 +08:00
SikongJueluo 3ba005b6cb test(ai-judge): gpt-5.6-luna qualification attempt (not qualified)
Three strict corpus-replay rounds against openai-codex/gpt-5.6-luna
(30000ms timeout), none qualified:

- run -01: 20/21, unclear-forward judged defer (expected deny)
- run -02: 20/21, conditional-preview judged allow (expected defer)
- run -03: 20/21, unclear-forward judged defer (expected deny)

unclear-forward failed in two of three runs -> systematic bias per the
two-rounds-same-case standard; conditional-preview failed once in the
permissive direction. No catalog entry added; all three reports
retained in reports/ as honest record (corpus NOT revised to
accommodate). Latency p50 4.9-5.9s, comparable to gpt-5.6-sol.
2026-08-21 23:41:26 +08:00
SikongJueluo 72c9366f07 test(ai-judge): deepseek v4 flash catalog qualification
Two strict corpus-replay rounds against deepseek/deepseek-v4-flash
(api openai-completions, 30000ms timeout):

- run -01: 20/21, covered-compound judged defer (non-repeating
  sampling miss, not systematic)
- run -02: 21/21, zero infrastructure failures, p50 1448ms /
  p95 1682ms / max 2146ms

Qualified. Adds the catalog entry (recommended, advisory data per
ADR 0008) and retains both reports; run -01 notes the single miss.
Full repo check+test green (236 + 48).
2026-08-21 23:41:26 +08:00
SikongJueluo 6001b014e0 refactor(ai-judge): move models-catalog.json to package root 2026-08-21 23:41:26 +08:00
SikongJueluo 8fef98e199 docs(ai-judge): rewrite README 2026-08-21 23:41:26 +08:00
SikongJueluo 7987fc7a3c feat(ai-judge): add advisory model catalog and strict corpus replay
- add versioned advisory model catalog shipped with the package and a fail-closed loader
- annotate the enforce session notice for untested, deprecated, and revoked models
- add --strict to corpus-replay with 0/1/2 exit codes and reject strict subset runs
- extract replay qualification into a pure module that recomputes matches and validates latencies
- remove the documented-but-unimplemented --thinking flag and stamp reports with a corpus version
- qualify gpt-5.6-sol as the first recommended entry and archive three real replay reports
- revise the corpus to 2026-08-21.2 changing unclear-forward expected defer to deny
2026-08-21 23:11:40 +08:00
SikongJueluo 061c60c344 feat(ai-judge): rework enforce mode as user-assumed-risk contract
- add config v2 with fixed judge model selection and fail-closed v1 enforce migration to shadow
- add built-in high-risk override for irreversible, publish, system, and credential shapes that always defers to the human
- remove the promotion gate, promotion records tool, and their tests
- fail closed with judge_model_unavailable when a configured judge model cannot be resolved
- notify once per session in enforce mode with the judge model and risk contract
- add ADR 0008 and update README and CONTEXT
2026-08-21 23:11:40 +08:00
SikongJueluo f4ba739878 docs(ai-judge): add package README
- document shadow and enforce usage, configuration, logs, and tools
- reorder the project authorizer chain to inner-cmd then ai-bash-judge
2026-08-21 23:11:34 +08:00
SikongJueluo 987d77a249 docs(ai-judge): v4 cohort declaration (-02 lapse, -03 fail, -04 PASS) and report (PIEXTENSIO-22) 2026-08-21 12:41:31 +08:00
SikongJueluo d57892d5ab feat(ai-judge): promotion-gate records seam and truth-table wiring (PIEXTENSIO-21) 2026-08-20 17:45:07 +08:00
SikongJueluo 45e91a700d fix(ai-judge): defer irreversible destructive operations regardless of intent (prompt v4, PIEXTENSIO-20) 2026-08-20 16:25:11 +08:00
SikongJueluo c5fbda5fea docs(ai-judge): record failed v3 promotion cohort (PIEXTENSIO-19) 2026-08-20 12:19:29 +08:00
SikongJueluo efc73c0f5c feat(ai-judge): prompt v3 few-shot calibration and corpus replay harness (PIEXTENSIO-18) 2026-08-19 23:45:47 +08:00
SikongJueluo 232bad549e feat(ai-judge): judge-owned audit log with local health gate (ADR 0006) 2026-08-19 23:45:43 +08:00
SikongJueluo 13cb19c349 docs(adr): enforce audit self-sufficiency, drop upstream host contract 2026-08-18 13:49:55 +08:00
SikongJueluo 6588773225 docs(testing): agent-driven TUI replay flow with 30-command cohort run 2026-08-18 00:33:47 +08:00
SikongJueluo 4ee25984b9 docs: acceptance checklist against PIEXTENSIO-3 contract
- record baseline, module map, category status, and pinned canonical revisions
- list owned residual gaps: upstream enrollment and human-decision seams, lifecycle ordering enumerations, negative v24 fixture
2026-08-17 22:38:38 +08:00
SikongJueluo 0637784f5f feat(ai-judge): conversation evidence with bounded whitelist capture
- add conversation.ts: compaction-aware active-branch capture, user-text-only whitelist, 16-item and 12,000-char bounds with latest-user preservation
- bump prompt to bash-shadow-v2 with explicit-user-intent authority rules and quoted untrusted intent evidence
- capture the requesting cwd and per-ask conversation state; flip evidence-quality flags from placeholders to measured values
- record the candidate-identity change for prior cohorts in the scenario-set doc
2026-08-17 22:25:06 +08:00
SikongJueluo 1d701ca0b0 feat(ai-judge): review sink with telemetry health and fail-closed truth table
- add review.ts sink adapter with session-start review-log toggle detection and a privacy key denylist enforced before delegation
- add judge.ts enforce truth table: allow requires mode, host contract, telemetry health, cohort qualification, owner approval, activation, judgment result, allow verdict, review acknowledgement, and current generation — each independently forces defer with a distinct reason
- route the authorizer callback through the sink and the v0.1 production gate state, which is structurally unreachable and therefore fail-closed
2026-08-17 21:37:48 +08:00
SikongJueluo 3f3bbb4c28 feat(ai-judge): capture model per permission request 2026-08-17 20:00:18 +08:00
SikongJueluo 0546a80497 feat(ai-judge): global config module with validation and cohort identity 2026-08-17 19:53:48 +08:00
SikongJueluo 0a1b9f259d build: align permission-system to 25.4 2026-08-17 19:20:49 +08:00
SikongJueluo 6c0e26bc6e fix(ai-judge): restore authoritative 15s default timeout 2026-08-17 19:17:40 +08:00
SikongJueluo 850f36c7a4 docs(research): archive shadow replay rounds and analyzer round-1 fixes
- rejoin round-1 rows hidden by terminal-event handling: normalize denied_with_reason, collapse forwarded double terminal rows, print quarantine counts, add --before window bound
- archive rounds 1-3 reports with blind-deny protocol, cross-round totals, and PIEXTENSIO-11 latency evidence
2026-08-17 18:59:19 +08:00
SikongJueluo ea7d93d63f docs: shadow evaluation methodology and round 1 archive
- add ADR 0005 reconstructing the PIEXTENSIO-9 comparison join from existing permission events with attribution rules and quarantine tripwires
- add the fixed replay scenario set with protocols and expected matrix, and archive the round 1 report and observations
2026-08-17 16:58:11 +08:00
SikongJueluo 92cabb6b7c fix(ai-judge): classify provider aborts after timeout as timeout 2026-08-17 16:58:10 +08:00
SikongJueluo 6407b7429a feat(ai-judge): offline shadow analyzer with reconstructed join 2026-08-17 16:58:10 +08:00
SikongJueluo 42f0a0aaab feat(permission): complete shadow review events for offline analysis
- key inner-cmd decisive review events by requestId so link decisions join offline
- record judge runtime id, prompt and tool schema versions, end-to-end and model latency, input and output usage, and evidence-quality flags on every judge result row
- record forwarded and session-mismatch preflight defers so they stay visible in the offline denominator
2026-08-17 16:58:10 +08:00
SikongJueluo 81f1da4100 chore: enable project-local ai-bash-judge smoke config 2026-08-16 23:54:58 +08:00
SikongJueluo f632fa34d1 fix(permission): raise verdict output budget for reasoning tokens 2026-08-16 23:54:58 +08:00
SikongJueluo 1afcbd3118 refactor(permission): consume structured bash payload
- require @gotgenes/pi-permission-system >=25.3.0 and read the complete local bash command from PromptPermissionDetails.payload instead of session-walking recovery
- remove the @sikongjueluo/pi-permission-shared package
- pass the triggering command unit to handlers via HandlerContext.unit in place of details.command
- add shadow-only AI judge modules for evidence projection, structured verdict requests, and prompt building, with vitest coverage
- record ADR 0004 and mark the ADR 0001 recovery mechanism superseded
- exclude pi-permission-system 25.3.0 from the pnpm minimumReleaseAge guard
2026-08-16 23:08:45 +08:00
SikongJueluo 6008c9e817 fix(pi-permission-inner-cmd): unwrap timeout in real-world command forms
- accept GNU timeout durations without a unit suffix and with decimals (timeout 240 …)
- detect the wrapper on details.command and strip it from the full command so scaffolded inputs (cd … && timeout … | tail) unwrap
- re-evaluate the full de-wrapped compound so sibling commands cannot hide behind the wrapper allow
- defer fail-closed when the unit is not a unique substring of the full command
- amend ADR 0001 with the relaxed grammar and the scaffolded-command handling
2026-08-12 11:20:46 +08:00
SikongJueluo 43ae2db90b feat(pi-permission-inner-cmd): defer xargs as a non-transparent wrapper
- add handlers/xargs.ts mirroring env: claim xargs-leading commands and defer
- register xargsHandler so leading-xargs commands log and defer instead of falling through silently
- add CONTEXT.md xargs example and ADR 0003 (xargs args come from stdin, so even the AI judge cannot know them)
2026-08-12 00:36:09 +08:00
SikongJueluo 5134e85d32 refactor(pi-permission-inner-cmd): dispatch commands through a handler registry
- replace the hardcoded timeout switch with an engine that iterates registered handlers
- extract the timeout logic into handlers/timeout.ts and add handlers/env.ts that defers env as non-transparent
- thread a partial-evidence bag so the engine exception log retains handler-derived values like innerCommand
- add CONTEXT.md with the transparent vs non-transparent wrapper glossary
- record ADR 0002: env always defers to the AI judge and is never unwrapped
2026-08-12 00:12:41 +08:00
SikongJueluo c00ae33031 build(deps): narrow pi-permission-system version range
- switch @gotgenes/pi-permission-system from >=24.0.0 to ^24.0.0
2026-08-11 22:03:01 +08:00
SikongJueluo 2014e7f793 refactor(pi-permission): extract shared bash-recovery package
- add @sikongjueluo/pi-permission-shared with recoverNativeBashCommand and its tests
- move the recovery module out of pi-permission-inner-cmd and import it from the shared package
- wire pi-permission-ai-judge to capture the UI-root session and recover the full bash command
- gate pi-permission-ai-judge registration on a UI-present root session
- add @types/node to pi-permission-ai-judge and allow its test script to pass with no tests
2026-08-11 21:53:46 +08:00
SikongJueluo 06b21a28a6 refactor(pi-permission-inner-cmd): narrow bash recovery uniqueness to one message
- walk entries in reverse and stop at the latest assistant message containing the id
- require the id to match exactly one block within that message rather than across the whole session
- resolve a cross-message id reuse to the latest call being authorized
- update ADR 0001 wording for the narrowed scope
- add a regression test for cross-message id reuse
2026-08-11 21:53:46 +08:00
SikongJueluo f21cf54ff1 fix(pi): install inner-cmd runtime dependency 2026-08-11 19:50:34 +08:00
SikongJueluo df1c9536e3 build(pi): register permission-inner-cmd extension
- add pi.extensions entry pointing to packages/pi-permission-inner-cmd/src/index.ts
- mark package private
2026-08-11 18:39:30 +08:00
SikongJueluo 24153412c9 feat(pi-permission-inner-cmd): authorize inner commands behind timeout wrappers
- recover the full bash command from the session by tool-call id
- add recognizer for the strict timeout wrapper grammar
- add authorizer mapping inner allow/ask/deny and forwarding agent name
- defer fail-closed on session mismatch, nested wrappers, and errors
- add unit tests for recovery, recognizer, authorizer, and lifecycle
- document the decision in ADR 0001
2026-08-11 16:33:08 +08:00
SikongJueluo c4d76ad284 feat(pi-permission-inner-cmd): scaffold inner-command permission package
- add package.json with pi extension entry and vitest setup
- add tsconfig and minimal session-start entry stub
- register package in pnpm lockfile
2026-08-11 16:33:07 +08:00
SikongJueluo 4ca585c915 docs(ai-bash-judge): add minimal-evidence research report
- analyze whether the proposed JudgeRequestV1 mixes evidence, adapter data, and transport metadata
- add Judgment Evidence and Execution Working Directory terms to CONTEXT.md
- ignore .workspace/ directory
2026-08-10 17:22:57 +08:00
SikongJueluo 5a67444188 docs: refine enforce mode and add judge participation term
- clarify enforce mode to approve only on allow verdicts
- add judge participation glossary term
2026-08-09 00:23:24 +08:00
SikongJueluo fba79504ac docs: add agent guidance and permission research docs
- add AGENTS.md and docs for issue tracker, triage labels, and domain docs
- add CONTEXT.md glossary for the permission authorization domain
- add research report on context ownership for forwarded bash asks
- ignore .pi-subagents and .codegraph directories
- remove pi-permission-ai-judge from settings packages
2026-08-09 00:23:24 +08:00
SikongJueluo f62a073de7 feat(pi-permission-ai-judge): add deferring authorizer extension
- add pnpm workspace scaffold with shared TypeScript config
- register an ai-bash-judge authorizer with pi-permission-system
- log permission request details and deterministic policy verdicts
- record review entries and defer to the next authorizer
2026-08-08 19:43:21 +08:00