Re: Conformance cases for mid-task halts

Hi Steven, Tymofii, Larry, and the CG,

This is an exceptional piece of architectural work. The "Evidence-Graded
Authorization Verdicts" draft (v0.5) is exactly the kind of rigorous,
implementable semantics the agent ecosystem needs right now.

I want to confirm the alignment mapping you laid out in Section 2.1:

· Quarantined: Maps perfectly to NTI's incomplete_check (fail-closed, MUST
NOT proceed).
· Insufficient Evidence: Maps to resume_basis=evidence, requiring
re-evidence to the required grade before resuming.
· Exemptions: The exemption_boundary MUST NOT override authority_revoked or
authority_not_determined.

Your E0-E4 operational grades (specifically E3 Reproducible and E4
Independently Verified) perfectly formalize what the Federation has been
doing over the last 72 hours.

On the test vector caveat (NTI §6.3 / PR #206):
You are correct that the initial PR #206 candidate vectors were static.
However, we have recently moved into active runtime verification.

· We have merged cross-project runtime checks for APS and Agent-DID
(conformance/agent-did-interop/).
· We have just merged a joint NTI-VATE "lost response" recovery fixture (PR
#65) with a real ML-DSA-87 signature and runtime verification script.
· The witness packages now bind canonicalized runtime test transcripts.

So while PR #206 remains a static vector set, NTI is actively generating
the E3 (Reproducible) evidence you are asking for.

The Linux Foundation Submission:
Steven, your work on this draft, alongside your role on the NTI TSC, is a
massive asset. As we finalize the LFDT proposal this week, I would like to
reference this W3C draft as an official interoperability artifact. It
proves NTI isn't just a standalone spec; it aligns cleanly with the W3C's
formal evidence-grading semantics.

Tymofii & Larry: This draft proves the CG is producing real, implementable
semantics for agent authorization.

Let's keep building the record.

Best,
Abisheak

On Sun, 8 Nov, 2026, 12:59 pm Xinghua Zhao, <zhaoxinghua07@126.com> wrote:

> Abisheak, Tymofii, Larry, all,
>
> Following the conformance thread on mid-task halts, and building on the
> runnable sidecar posted on 30 October, here is v0.5 of the evidence-graded
> authorization draft. It incorporates the external-model review (five
> MUST-FIX items, then four non-blocking follow-ups) and the public-archive
> corrections.
>
>   "Evidence-Graded Authorization Verdicts for Agent Protocols" (v0.5)
>
> What is new / changed, which I would like the CG to pull/capture:
>
> 1. Explicit NTI alignment (not adoption): quarantined maps to
> incomplete_check and MUST NOT proceed (fail-closed); insufficient_evidence
> maps to resume_basis=evidence; exemptions carry an exemption_boundary and
> MUST NOT be used for authority_revoked or authority_not_determined. NTI
> state-machine references are to NTI section 6 (not section 3). NTI section
> 6 itself notes that this state machine is drawn from a transition-contract
> draft that is "explicitly not agreed by either side" and "not a ratified
> normative annex"; this draft therefore claims alignment only.
> 2. Operational E0-E4: E2 = an outside observer can confirm a public
> record; E3 = a party other than the author re-runs the same inputs and
> obtains the same digest; E4 = a non-affiliated party (no employment, no
> equity, not the same natural person) re-runs and signs a detached signature
> over the receipt digest.
> 3. Binding fields: object_sha256, object_version, environment,
> validity_window, evaluator_identity, evidence_status (closed set:
> SUFFICIENT / INSUFFICIENT / INVALID); plus resume_basis/exemption_boundary
> on the receipt.
> 4. Conformance tests state assertion, clock/observation window, and a
> negative control for each of: tryLater vs timeout, T-30/T-0, grade
> monotonicity, scope binding, quarantine fail-closed.
> 5. Implementation availability with evidence: the 30 October sidecar was
> anonymously re-downloaded on 2026-11-08 (sha256 matches the sent original)
> and its --selftest passes across three registry modes; this validates the
> receipt fields at the level of that selftest, not a full protocol
> implementation. NTI section 6.3 notes that the PR #206 candidate vectors
> have not been run against any runtime; that is separate from this selftest.
> 6. Status and limits: discussion draft, not adopted and not endorsed; CC
> BY 4.0.
>
> The full v0.5 text is below. If the CG prefers an issue/PR/wiki, I can
> move it there. I am not claiming consensus or adoption; I am asking whether
> these sections are useful input for the record.
>
> Best regards,
> Xinghua Zhao
> Individual CLA commitment - no organizational affiliation
> ORCID: 0009-0001-0512-1237
>
> ---
> FULL DRAFT (v0.5)
> ---
>
> # Evidence-Graded Authorization Verdicts for Agent Protocols
>
> **Status: Discussion Draft / Input to the CG — NOT adopted, NOT endorsed
> by the CG or any organization.**
> **Version:** 0.5 · **Date:** 2026-11-08
> **Author:** Xinghua Zhao (Individual CLA commitment; no organizational
> affiliation) · ORCID 0009-0001-0512-1237
> **License:** CC BY 4.0
>
> > This draft represents the author's personal view as an individual
> expert. It does not represent any employer, company, or organization, and
> it does not claim CG consensus or adoption.
>
> ## Abstract
>
> Agent protocols increasingly separate **authorization decisions** from
> **execution**. When an enforcement point pauses, denies, or allows an agent
> action, the record of *why* is often incomplete, unverifiable, or not
> comparable across implementations. This draft proposes a minimal,
> implementable **evidence-grade dimension** for authorization verdicts: each
> verdict carries (a) a grade describing the quality of its supporting
> evidence, (b) a four-state outcome, and (c) a policy/scope declaration
> bounding what the verdict establishes. It **extends** existing
> agent-protocol constructs (pause/decision receipts, incomplete checks,
> `tryLater` responses, finish-or-stop contracts); it does not replace them.
>
> ## 1. Problem
>
> 1. Binary prose, graded practice: an allow after a complete check and an
> allow after a timeout are often recorded identically.
> 2. Incomplete checks are invisible: “checked and authorized” vs “not
> checked” is indistinguishable in the record.
> 3. No common scope language: a PASS is quoted outside the
> version/environment/test scope it was produced under.
> 4. Unverifiable evidence: logs/hashes/versions are frequently absent, so a
> verdict cannot be independently reproduced.
>
> ## 2. Relationship to prior work (increment, not duplication)
>
> Reference point in the NTI line: the state machine and `incomplete_check`
> block are in **NTI §6** (not §3; §3 is the five pillars). NTI §6 itself
> records that this state machine and the `resume_basis` field are drawn from
> a transition-contract draft that is **"explicitly not agreed by either
> side"** and **"not a ratified normative annex"**; this draft therefore
> claims **alignment with a working reference only, not NTI adoption**. The
> increment here is the **evidence-quality axis** and **scope binding**, not
> a new state machine.
>
> ### 2.1 Explicit mapping (alignment, not adoption)
>
> | This draft | NTI construct | Rule |
> |---|---|---|
> | `authorized` | check completed + policy satisfied | fail-closed check
> required |
> | `denied` | check completed + policy not satisfied | fail-closed check
> required |
> | `quarantined` | `incomplete_check` | **MUST NOT proceed** while
> quarantined; fail-closed; retryable |
> | `insufficient_evidence` | `resume_basis` = evidence | resume only after
> re-evidence to the required grade |
> | exemption path | `resume_basis` = exemption | allowed only where policy
> permits; **MUST NOT** be used for `authority_revoked` or
> `authority_not_determined` |
>
> A2A/MCP task-state mapping is **not yet completed**; this draft does not
> claim one.
>
> Note on test vectors: the NTI §6.3 reference to PR #206 states that its
> candidate vectors have **not been run against any runtime** and are "not a
> conformance result". That is separate from the selftest evidence in §2.2,
> which is this sidecar's own run.
>
> ## 2.2 Implementation availability
>
> A runnable reference sidecar was posted publicly on 2026-10-30
> ("long-running call: runnable sidecar, PAUSE-receipt fields, two
> finish-or-stop contracts",
> https://lists.w3.org/Archives/Public/public-agentprotocol/2026Oct/0108.html).
> Anonymous re-download of the attachment (2026-11-08) hashes identically to
> the sent original (sha256 `82444a4b…`), and `python transition_sidecar.py
> --selftest` reports:
>
> ```
> SELFTEST across registry modes
>   [PASS] reachable   trigger=revocation           C.admitted=False
> replay=4/4
>   [PASS] unreachable trigger=registry_unreachable C.admitted=False
> replay=6/6
>   [PASS] stale_root  trigger=stale_root           C.admitted=False
> replay=6/6
> SELFTEST PASS (0 failure(s))
> ```
>
> This exercises the receipt fields and finish-or-stop checks below **at the
> level of the sidecar's own selftest**; it is not a full protocol
> implementation and does not validate other implementations.
>
> ## 3. Evidence grades (E0–E4)
>
> | Grade | Meaning | Operational test |
> |---|---|---|
> | E0 | Assertion only | no artifact, no reproducible record |
> | E1 | Declared | self-reported configuration/policy, no independent
> observation |
> | E2 | Observable | an outside observer can read a public record (log,
> endpoint, receipt) and confirm the stated event |
> | E3 | Reproducible | a party **other than the artifact's author** re-runs
> the same inputs in the declared environment and obtains the same outputs
> (same digest) |
> | E4 | Independently verified | a non-affiliated party (see below) re-runs
> and **signs** the result; signature mechanism per §3.1 |
>
> Rules:
>
> - A verdict's grade is the **minimum** grade of its supporting evidence;
> no weighting.
> - **E4 non-affiliation** means all three: no employment relationship, no
> equity/ownership, and not the same natural person operating another seat.
> - Grades bound claims; they are not marketing levels.
>
> ### 3.1 E4 signature mechanism (minimal)
>
> The verifier signs a **detached signature over the canonical receipt
> digest**. Required fields in the E4 attestation: `evaluator_identity`,
> `signature_algorithm`, `signed_digest`, `validity_window`. Choice of
> algorithm and trust anchors is an open question (§8), not fixed by this
> draft.
>
> ### 3.2 Binding fields (MUST)
>
> Every verdict MUST bind to: `object_sha256`, `object_version`,
> `environment`, `validity_window`, `evaluator_identity`, `evidence_status`.
>
> `evidence_status` is a closed set: `SUFFICIENT` (evidence meets the policy
> grade), `INSUFFICIENT` (evidence present but below the policy grade),
> `INVALID` (evidence cannot be verified or has been superseded).
>
> ## 4. Verdict states
>
> | State | Meaning | Retryable | Resume basis |
> |---|---|---|---|
> | `authorized` | check completed, policy satisfied, evidence attached | —
> | — |
> | `denied` | check completed, policy not satisfied | — | — |
> | `quarantined` | check **incomplete**; MUST NOT proceed | yes | new check
> |
> | `insufficient_evidence` | decision made, evidence grade below policy
> requirement | yes | `evidence` |
>
> Where an exemption exists, it MUST be recorded as `resume_basis=exemption`
> with an explicit `exemption_boundary`; exemptions MUST NOT apply to
> `authority_revoked` or `authority_not_determined`.
>
> **Compatibility mode:** `insufficient_evidence` MAY be carried as an
> annotation on `denied` for implementations that only support three states.
>
> ## 5. Pause/decision receipt fields
>
> Where a pause/decision receipt is emitted (CG term: PAUSE receipt), it
> SHOULD include:
>
> - `evidence_grade` (E0–E4) and `evidence_status`
> - `verdict_state`
> - `policy_id` / `policy_version` / `scope` (version + environment + test
> scope)
> - `evidence_refs` (hashes, log refs, provenance)
> - `evaluator_identity`, `object_sha256`, `object_version`, `environment`,
> `validity_window`
> - `resume_basis` (`evidence` | `exemption`) and `exemption_boundary` (when
> applicable)
> - `reevaluate_on` (event or condition requiring a new decision)
>
> An incomplete check MUST be visible in the receipt and MUST NOT be
> silently converted to `denied` or `authorized`.
>
> ## 6. Conformance tests (assertion + window + negative control)
>
> 1. **tryLater vs timeout.** Assertion: `tryLater` is recorded as a
> response; timeout/no-response is recorded as observed failure. Window: same
> admission window; clock = virtual or declared. Negative control: responder
> returns `tryLater` but receipt records timeout → FAIL.
> 2. **T-30 / T-0.** Assertion: after T-30 no new call is admitted; admitted
> calls are evaluated against their declared finish-or-stop contract. Window:
> `T-30` and `T-0` measured from the same declared base timestamp
> (`decision_time`). Negative control: a call admitted after T-30 → FAIL.
> 3. **Grade monotonicity.** Assertion: `verdict.grade ≤ min(evidence
> grades)`. Window: N/A (assertion evaluated on the produced receipt; no
> timing dimension). Negative control: verdict grade E3 with one supporting
> evidence E1 → FAIL.
> 4. **Scope binding.** Assertion: a verdict quoted without `scope` is
> non-conforming. Window: N/A (static field check). Negative control: removal
> of `scope` → FAIL.
> 5. **Quarantine fail-closed.** Assertion: while `quarantined`, no
> execution proceeds. Window: the quarantine interval recorded in the
> receipt. Negative control: execution proceeds during quarantine → FAIL.
>
> Expected outputs are digest comparisons where the sidecar emits JSON
> (`--json out.json`); each test states its expected digest or field value.
>
> ## 7. Security and privacy
>
> - Do not embed secrets or personal data in evidence refs; use
> hashes/opaque refs.
> - Evidence grades characterize verifiability, not truth of underlying
> claims.
> - Quarantine/exemption paths must be auditable without exposing sensitive
> payloads.
>
> ## 8. Open questions
>
> **Decision:**
> 1. Should `quarantined` be a first-class receipt state, or a derived
> annotation on `incomplete_check`?
> 2. What minimum grade should a given use case require (e.g., payments vs.
> non-payments)?
>
> **Technical:**
> 3. E4 signature algorithm and trust anchors (COSE/JWS vs. detached CMS)?
> 4. Mapping table granularity for existing receipt formats — per-field or
> per-state?
> 5. Is `reevaluate_on` better expressed as a condition or an event type?
>
> ## 9. References (public archive; Date headers used)
>
> 1. Xinghua Zhao, "long-running call: runnable sidecar, PAUSE-receipt
> fields, two finish-or-stop contracts", **Fri, 30 Oct 2026 13:33:59 +0800**.
>
> https://lists.w3.org/Archives/Public/public-agentprotocol/2026Oct/0108.html
> 2. Larry Lewis, "Re: Revocation vocabulary and runtime halt",
> **2026-10-29** (incomplete registry check visible in PAUSE receipt;
> tryLater vs timeout recorded separately; T-30/T-0 tested independently).
>
> https://lists.w3.org/Archives/Public/public-agentprotocol/2026Oct/0090.html
> 3. Abisheak P, "Re: Four open questions on the Agentic Web report",
> **2026-10-29** (`incomplete_check` block with reason: tryLater,
> not_authoritative, or internal_error; fail closed with PAUSE).
>
> https://lists.w3.org/Archives/Public/public-agentprotocol/2026Oct/0091.html
> 4. Background thread: "Conformance cases for mid-task halts", starting
> **2026-11-01** (Abisheak P et al.).
>
> https://lists.w3.org/Archives/Public/public-agentprotocol/2026Nov/0008.html
> 5. NTI-1 thread: original by **Abisheak P**, reply by **Larry Lewis**
> (2026-11-05); Larry describes himself as "contributor pending explicit
> acceptance".
>
> https://lists.w3.org/Archives/Public/public-agentprotocol/2026Nov/0052.html
> 6. NTI-1 original post: Abisheak P, "NTI-1 Specification: CI Green, Ready
> for Contributor Onboarding & AI Agent Integrations", 2026-10-30.
>
> https://lists.w3.org/Archives/Public/public-agentprotocol/2026Oct/0111.html
> 7. NTI-1 follow-up: Abisheak P, same subject, 2026-11-05.
>
> https://lists.w3.org/Archives/Public/public-agentprotocol/2026Nov/0053.html
>
> No claim of endorsement by any CG participant or organization.
>
> ## Disclaimer
>
> This is a discussion draft. It does not claim W3C CG consensus, adoption,
> or endorsement.
>
> ## Change log
>
> - 0.5 (2026-11-08): closed the four non-blocking items from the P5-re
> receipt: evidence_status enum, per-test windows for tests 3-5, original
> NTI-1 links (Oct/0111, Nov/0053); no semantic change to verdicts or grades.
> - 0.4 (2026-11-08): NTI §6 mapping table (fail-closed,
> resume_basis/exemption boundary); E2/E3 boundary and E4 signature
> mechanism; binding fields; conformance assertions/windows/negative
> controls; references corrected (mid-task halts added, 0052
> attribution/date); selftest output quoted.
> - 0.3 (2026-11-08): implementation availability (0108 sidecar); NTI
> incomplete_check reason mapping; archive URLs.
> - 0.2 (2026-11-08): Discussion Draft; increment table; E4 definition; min
> rule; compatibility mode; observable criteria; ORCID; disclaimer.
> - 0.1 (2026-11-08): initial draft.
>
>

Received on Sunday, 8 November 2026 08:05:06 UTC