- From: Abisheak P <abisheakp197@gmail.com>
- Date: Sun, 25 Oct 2026 18:55:10 +0530
- To: public-agentprotocol@w3.org
- Message-ID: <CALwwdfqJaavLaaapAJ7Hig-KAcrtPhhxkG8ReTzP6x-GPdF6sA@mail.gmail.com>
Hi everyone, Following the deep architectural discussions on authority, revocation, and fail-closed states, I realized that theoretical debate has a bottleneck. To move this forward, we need empirical, reproducible data. To that end, we have deployed the NTI Public Compliance Leaderboard. It is a fully automated, self-sustaining benchmark engine for AI agent security. How it works: 1. Every day, a GitHub Action automatically clones the top frameworks (LangChain, CrewAI, AutoGen, LlamaIndex) and runs nti-scanner. 2. The scanner performs static AST analysis against the NTI-1 standard (checking for signed tool calls, fail-closed handoffs, and Merkle audit trails). 3. It generates a post-quantum attestation (ML-DSA-5) for every score, which is publicly verifiable offline. 4. The data feeds directly into the Next.js leaderboard. Neutrality & Dogfooding: To ensure absolute neutrality, NTI is not hardcoded to the top. Our own Rust core (Neutral-Trust-Infrastructure) is included in the daily automated scan. It currently scores 100/100 (A+) mathematically, just like any other repo would. Current Real-World Results: · NTI Reference Implementation: 100/100 (A+) · aps-conformance-suite: 66/100 (D) (This is a live scan of the repo mentioned in the recent threads) · AutoGen: 25/100 (F) · LangChain: 16/100 (F) · CrewAI: 0/100 (F) You can view the live cryptographic proofs, AST findings, and rankings here: https://nti-leaderboard.vercel.app We built a 1-click "Submit" flow. If any group member wants their protocol or framework added to the daily automated scan, you can submit it, merge the PR, and the engine will handle the rest. We believe this provides the concrete data needed to accelerate the W3C standardization process. Best, Abisheak
Received on Sunday, 25 October 2026 13:25:26 UTC