How AI Crawlers Are Straining Web Infrastructure

I'm sharing a public document sent to the AIPREF list by its author, Audrey Hingle.

Best regards
Laurent

---
Hi All, 

I'm sharing a paper I wrote as part of my Internet Society Fellowship completed last month, “Getting Bots to Respect Boundaries: How AI Crawlers Are Straining Web Infrastructure.” 

I thought it might be useful context for AIPREF because it documents what publishers are doing right now to stay online under AI-driven crawling strain. I also looked at how these mitigation efforts could interact with new protocols. 

PDF here: https://internet.exchangepoint.tech/content/files/2026/02/Getting-Bots-to-Respect-Boundaries.pdf

Best,
Audrey