Measuring conformance to agent protocols, and a proposed group

Hi all,

Just joined. One offer and one ask.

The offer: we run a deterministic audit of what web resources actually
expose to a non-browser agent. 36 checks, no model in the loop, evidence
recorded for each one. Across 30 real production sites the median scored
61.75 out of 100. A protocol is only as good as what deployments actually
do with it, and we can measure that. Happy to run the cohort against
anything this group specifies.

The ask: I have proposed a Community Group for the measurement side, Agent
Conformance and Benchmarking. Conformance reporting formats, runnable test
suites with adversarial fixtures, versioned corpora. Built to consume this
group's output, not compete with it.

https://www.google.com/url?q=https://www.w3.org/community/blog/2026/08/25/proposed-group-agent-conformance-and-benchmarking-community-group/&source=gmail&ust=1787766514570000&sa=E

It needs five supporters to launch. Support button:
https://www.google.com/url?q=https://www.w3.org/community/groups/proposed/%23agent-conformance&source=gmail&ust=1787766514570000&sa=E

Julian Joseph
ApexClaw