mirror of
https://github.com/SikongJueluo/pi-extensions.git
synced 2026-10-05 20:02:55 +08:00
SikongJueluo
850f36c7a4
- rejoin round-1 rows hidden by terminal-event handling: normalize denied_with_reason, collapse forwarded double terminal rows, print quarantine counts, add --before window bound - archive rounds 1-3 reports with blind-deny protocol, cross-round totals, and PIEXTENSIO-11 latency evidence
75 lines
4.2 KiB
Markdown
75 lines
4.2 KiB
Markdown
---
|
|
status: accepted
|
|
---
|
|
|
|
# Reconstruct the Shadow comparison join from existing permission events
|
|
|
|
PIEXTENSIO-9 defines the v0.1 evaluation contract: for each enrolled
|
|
permission request, correlate the Judge's prediction with the later human
|
|
decision, offline, by `requestId`. Its required event stream is
|
|
`authorizer_link.invoked` (enrollment), `ai_bash_judge.result` (prediction),
|
|
and `permission_request.human_decided` (blind reference). None of these
|
|
events exist verbatim in `@gotgenes/pi-permission-system` 25.3/25.4, and the
|
|
tickets acknowledge this: until the upstream gaps close, collected records
|
|
are **diagnostic Shadow evidence only**, never promotion-grade.
|
|
|
|
We decided the offline analyzer reconstructs the contract from the events
|
|
that *do* exist, rather than waiting for upstream changes (or patching the
|
|
dependency).
|
|
|
|
## Reconstruction rules
|
|
|
|
| Contract event | Reconstructed from | Why it holds |
|
|
| --- | --- | --- |
|
|
| Enrollment | `authorizer_chain_resolved` whose `links` array contains `ai-bash-judge` | Upstream records resolved links **before any link runs** — its own doc comment cites exactly the "judge never ran vs. ran and deferred" distinction. Appending after `authorizer_link.invoked` semantics. |
|
|
| Prediction | `ai_bash_judge.result` | Judge-owned; already keyed by `requestId`. |
|
|
| Human decision | `permission_request.approved`/`.denied` **with attribution** | `approved_for_session`/`approved_for_serving_session` can only originate from the human (upstream `decideFromVerdict` grants links only the one-shot `approved` state). A plain `approved`/`denied` is attributed to the human **unless** the same `requestId` carries a decisive link marker (`inner_cmd.allow`/`inner_cmd.deny`, made joinable in this repo). |
|
|
|
|
`permission_request.session_approved` and
|
|
`permission_request.infrastructure_auto_allowed` are not enrollments: the
|
|
chain never runs for rule-satisfied or session-remembered asks, so they are
|
|
correctly outside the denominator.
|
|
|
|
Two further upstream behaviors are encoded as rules (round 1 findings):
|
|
|
|
- Terminal resolutions `denied` and `denied_with_reason` (upstream's
|
|
provide-reason deny) both normalize to human deny.
|
|
- The forwarded decision path double-writes the terminal row with the same
|
|
`requestId` and resolution in adjacent file order. Identical-resolution
|
|
duplicates are collapsed to the first row; **conflicting** resolutions
|
|
remain quarantined as `multiple_human_decisions` — the analyzer must
|
|
never pick a convenient outcome among alternatives (PIEXTENSIO-9).
|
|
|
|
Rows where attribution fails (a plain `approved` sharing the request with a
|
|
link allow — the one case the marker rule cannot decide) stay joined but are
|
|
marked `unproven` and never enter the comparison matrix. Duplicate results,
|
|
result-before-enrollment ordering violations, multiple human decisions, and
|
|
unreadable terminal states are quarantined by category, never dropped.
|
|
|
|
## Alternatives rejected
|
|
|
|
- **Upstream PR now**: the three events plus a write-acknowledgement seam
|
|
are structurally required *for promotion*, but upstream merge latency and
|
|
release cadence would stall cohort collection. The reconstruction gives
|
|
the calibration data that a future PR needs as justification.
|
|
- **Local patch of the dependency**: maintains a fork against a moving
|
|
25.x; the global installation loads released versions.
|
|
|
|
## Consequences
|
|
|
|
- The analyzer reads upstream implementation details (event names, state
|
|
vocabulary, link-state semantics), not a contract. An upstream refactor
|
|
can silently break the reconstruction; the quarantine categories and
|
|
coverage metrics are the designed tripwire — a spike in
|
|
`terminal_event_unreadable` or a coverage collapse indicates drift, not
|
|
data.
|
|
- `¬marker ⇒ human` is an inference. PIEXTENSIO-9 forbids analyzers from
|
|
inferring outcomes; diagnostic reports therefore carry an explicit
|
|
`DIAGNOSTIC (reconstructed join; not promotion-grade)` header, and the
|
|
promotion floor (PIEXTENSIO-10) cannot be satisfied from reconstructed
|
|
rows at all.
|
|
- Write-acknowledgement is likewise reconstructed only negatively: a
|
|
disabled review sink is detected from the config file, but an
|
|
in-flight write failure surfaces only as a missing result (a coverage
|
|
gap), not a positive fault signal.
|