Subject: Empirical Benchmark Data: Automated AST scans for Agent Frameworks

Hi everyone,

To add concrete data to our ongoing conformance discussions on authority,
revocation, and trust, we built an automated static analysis (AST) pipeline
for AI agent frameworks.

Instead of debating theory, we decided to measure the current state of the
industry against the NTI-1 cryptographic standards.

How it works:
A GitHub Action runs daily, clones the top frameworks, runs nti-scanner to
check for signed tool calls and fail-closed execution states, and generates
a cryptographically signed attestation (ML-DSA-5) for each score.

The Current Reality (Live Data):

· NTI Reference Implementation: 100/100 (A+)
· aps-conformance-suite (W3C): 66/100 (D) (1,288 findings)
· AutoGen: 25/100 (F)
· LangChain: 16/100 (F)
· CrewAI: 0/100 (F)

You can view the live cryptographic proofs, AST findings, and rankings here:
👉 https://nti-leaderboard.vercel.app

We included the aps-conformance-suite in our daily scan because we believe
the standard itself must be held to the same cryptographic rigor as the
frameworks it aims to secure. It currently scores a 66 because the test
vectors do not yet include offline-verifiable post-quantum signatures.

We have also filed a private security advisory with the CrewAI team to help
them remediate their unsigned tool-call execution loop.

To prove this isn't a black box: anyone can run the scanner locally (pip
install nti-scanner), and any group member can submit their repository to
be added to the automated daily pipeline.

We look forward to seeing how these numbers evolve as the W3C standard
matures.

Best,
Abisheak