Subject: Live Benchmark: Automated AST scanning and cryptographic attestations for Agent Frameworks

Hi everyone,

Following the deep architectural discussions on authority, revocation, and
fail-closed states, I realized that theoretical debate has a bottleneck. To
move this forward, we need empirical, reproducible data.

To that end, we have deployed the NTI Public Compliance Leaderboard. It is
a fully automated, self-sustaining benchmark engine for AI agent security.

How it works:

1. Every day, a GitHub Action automatically clones the top frameworks
(LangChain, CrewAI, AutoGen, LlamaIndex) and runs nti-scanner.
2. The scanner performs static AST analysis against the NTI-1 standard
(checking for signed tool calls, fail-closed handoffs, and Merkle audit
trails).
3. It generates a post-quantum attestation (ML-DSA-5) for every score,
which is publicly verifiable offline.
4. The data feeds directly into the Next.js leaderboard.

Neutrality & Dogfooding:
To ensure absolute neutrality, NTI is not hardcoded to the top. Our own
Rust core (Neutral-Trust-Infrastructure) is included in the daily automated
scan. It currently scores 100/100 (A+) mathematically, just like any other
repo would.

Current Real-World Results:

· NTI Reference Implementation: 100/100 (A+)
· aps-conformance-suite: 66/100 (D) (This is a live scan of the repo
mentioned in the recent threads)
· AutoGen: 25/100 (F)
· LangChain: 16/100 (F)
· CrewAI: 0/100 (F)

You can view the live cryptographic proofs, AST findings, and rankings
here: https://nti-leaderboard.vercel.app

We built a 1-click "Submit" flow. If any group member wants their protocol
or framework added to the daily automated scan, you can submit it, merge
the PR, and the engine will handle the rest.

We believe this provides the concrete data needed to accelerate the W3C
standardization process.

Best,
Abisheak

Received on Sunday, 25 October 2026 13:25:26 UTC