Re: ATP CG update — four focused requests for review and contributions

Hi Larry,

Since the shared family isn't drafted yet, I ran the check on the
closest existing material, the 20 Trust Credential status cases (item
7) in PR #6, and turned what I found into questions on the admission
rules. This is read from the spec text, I didn't run any code.

Main finding: 17 of the 20 cases (02-04 and 06-19), plus the first
step of 05, expect a denial. A verifier that denies everything fails
only 01, the second step of 05, and 20. Cases 06-19 differ in input
but share one expected output, so an implementation that fails closed
for the wrong reason passes all of them. Paul's proposal in PR #6 to
carry the VC-BSL errorType in each expected block makes these cases
distinguishable. I'd treat it as required, not optional. Lars was
first to raise recording the reason for denial.

Notes on the cases:

- Each negative case should be stated as a one-field change from
status-01. status-14 catches a verifier that skips the list proof only
if the fixture's bits are 0/0. If a bit is set, the verifier denies
for the bit and passes anyway. Same for 15.
- status-11 bundles four conditions (negative, JSON number,
non-decimal, out-of-range) under one ID, and status-13 bundles timeout
and error. One vector per condition, otherwise an implementation that
catches one of them passes.
- status-08 and 09 should specify both variants (only revocation
present / only suspension present; duplicate revocation / duplicate
suspension). If the vector drops the revocation entry, a verifier that
checks only for revocation still rejects and passes. The variant that
drops only suspension is the one that catches it.
- status-17 has no boundary positive. A list of exactly 131072 entries
should pass. Without it, a threshold off by a unit (bytes vs bits) is
not detected.
- status-05 needs two status lists at two moments, but the vector
shape has one resources map and one evaluationTime. There is also no
twin case: revocation set, then a newer list with it cleared. Paul's
point that the issuer, not the verifier, has to hold that rule decides
what that case can expect.
- status-20 has no negative partner: a stapled list that is stale,
signed for another list id, or fails its proof.
- Only status-03 has a full expected block. For 06-19 the values of
statusConformant and credentialValid are not stated, e.g. whether a
credential with a retrieval error is credentialValid true or false.

On external effect, which you asked me to flag: every expected field
in these cases is an output of the verifier itself. None observes a
relying party actually refusing an action. For a
delegation-and-revocation family that is the part that matters.

Questions on the admission rules:

1. Is a case admitted only with a positive and a negative control that
differ in one stated input and have different expected outputs?
2. Should every proposed family be run against two reference stubs,
allow-all and deny-all, with the rule that each case must fail at
least one and the family must fail both?
3. What exactly is revoked: the delegator's credential, the
delegate's, or the grant itself? Is revoking a middle link in a
re-delegation chain in scope?
4. Timing: is there a case where the delegate acts after revocation
but inside a cached-status window, and what is the expected outcome
then?
5. Does the expected outcome include the relying party's observable
response, not only the verifier's decision?
6. Should the run record contain errorType, the id and verified
instant of the status list used (Lars's sentence), fixture hashes, and
implementation and runner versions?

I'll review the draft family against the same list once it's posted.

Evgenii

On Tue, Nov 3, 2026 at 5:45 AM Larry Lewis <llewis@sovrlabs.com> wrote:
>
> Hi Evgenii,
>
> Thank you—this is a useful contribution, and I’ll record you as a volunteer reviewer for item 4.
>
> Your focus on whether each case can actually fail fits what we need. Please check the expected difference between the positive and negative controls, whether a deliberately defective implementation is detected, and what evidence in the run record supports that distinction.
>
> We have experimental test material, but the proposed shared delegation-and-revocation test family still needs its admission rules and evidence requirements consolidated. Please send your checklist as questions on those rules first. That will help us shape the draft before asking you to review it.
>
> Please also flag ambiguous outcomes, missing observations, or cases that demonstrate only an internal decision without establishing an external effect. Posting your notes to the list will give other members a chance to contribute.
>
> Thank you for offering an asynchronous written review within the response window.
>
> Best,
> Larry Lewis
> Chair, Agent Trust Protocol Community Group
> SovrLabs
>
>
> From: Evgenii Arsentev <arsevgen22@gmail.com>
> To: "Larry Lewis"<llewis@sovrlabs.com>
> Cc: <public-atp@w3.org>, <chgaowei@gmail.com>, <xusong@migu.chinamobile.com>
> Date: Mon, 02 Nov 2026 09:31:28 -0600
> Subject: Re: ATP CG update — four focused requests for review and contributions
>
> Hi Larry,
>
> Item 4, review.
>
> My contribution is a written review of the proposed
> delegation-and-revocation test family, built around one question - can
> each case actually fail? For every case I'll check whether there's a
> pair of controls, one positive and one negative, that give different
> results, and which evidence in the run record shows that. Cases where
> both controls pass the same way I'll flag separately, because they
> prove nothing.
>
> On background, I'm one of the editors of the v0.1 reporting format in
> the W3C Agent Conformance and Benchmarking CG. I also wrote a preprint
> on this exact problem, a test suite that can't tell conditions apart
> (Qeios, DOI 10.32388/0BV3Z8).
>
> I work async and in writing. The first review will come within your
> seven-day window, as soon as the draft test family is available. If
> there's no draft yet, I can send the same checklist as questions on
> the admission rules. I'll post the notes to the list.
>
> Best,
> Evgenii
>
>
> On Mon, Nov 2, 2026 at 1:37 PM Larry Lewis <llewis@sovrlabs.com> wrote:
> >
> >
> > Hello everyone,
> >
> > I would like to bring our attention back to completing ATP’s four specification tracks: did:atp, Trust Scoring & Credentials, Privacy-First Interaction, and Conformance & Interoperability.
> >
> > Our collaboration with the AI Agent Protocol community and the NTI work is helping us identify useful questions and tests. I want those findings to support ATP’s own deliverables while we keep implementation results, community proposals, and adopted group decisions clearly distinguished.
> >
> > I am asking for help with four focused work items:
> >
> > 1. Credential status and proofs. Please review issue #3 and draft PR #6, particularly revocation, suspension, and failure handling. An experimental signed-fixture package now exists, but review found two validation gaps that need correction before wider reproduction. We need implementer input on the credential securing profile and a separate implementation to reproduce the corrected cases.
> > https://github.com/w3c-cg/atp/issues/3
> > https://github.com/w3c-cg/atp/pull/6
> >
> > 2. Trust scoring and issuer authority. We need a small, reproducible shared-operator test with clear cohort, timing, and score-ceiling inputs. I welcome volunteers to help define the inputs and independently check the output. This follows through on our earlier scoring work without reopening completed issues unnecessarily.
> >
> > 3. Identity and privacy alignment. I would like us to resume the discussion with Song Xu and Gaowei Chang and arrange our proposed working session. The current Agent Authorization PR #46 has moved to an HTTPS Client ID Metadata Document (CIMD) design with an explicit DID binding; it remains an open proposal, and its detailed Verifiable Credential profile is still reserved. We should review how ATP’s identity, trust credentials, privacy, and revocation work can complement that design, and document any gaps before proposing shared tests. This work will coordinate with item 4 rather than create a separate overlapping test effort.
> > https://github.com/w3c-cg/ai-agent-protocol/pull/46
> >
> > 4. Conformance and interoperability. Let us agree on one bounded delegation-and-revocation test family, including the admission rules, expected outcomes, positive and negative controls, and evidence needed. We need volunteers for implementation, reproduction, and review. External experiments will be attributed to their authors and operators; they will not be presented as ATP CG findings without group review.
> >
> > Please reply within seven days of this message with:
> >
> > ● The item number you can help with;
> >
> > ● One concrete contribution you can make;
> >
> > ● Your availability or a realistic completion estimate.
> >
> > Specification review, test design, documentation, and independent reproduction are all valuable contributions. We can continue asynchronously while arranging a short working session.
> >
> > As chair, I will coordinate the work, track volunteers and open questions, and bring proposed decisions back to the group. Thank you for helping us turn these drafts into reviewed, reproducible community deliverables.
> >
> > Larry Lewis
> > Chair, Agent Trust Protocol Community Group
> > Sovr INC
> >
> >
> >
>
>
> --
> Evgenii Arsentev, PhD
> Chief Executive Officer, digital health
> ORCID 0000-0002-9120-7298 · https://arsentev.ai
>
>
>


-- 
Evgenii Arsentev, PhD
Chief Executive Officer, digital health
ORCID 0000-0002-9120-7298 · https://arsentev.ai

Received on Tuesday, 3 November 2026 17:25:19 UTC