@osm_tech I'm going to resist the urge to say something really unpleasant but I will say this.
It's the open internet that suffers from the VC-backed LLM / AI bubble. They could *not* care less.
"If you think these crawlers respect robots.txt then you are several assumptions of good faith removed from reality." - Drew DeVault, https://drewdevault.com/2025/03/17/2025-03-17-Stop-externalizing-your-costs-on-me.html