Claim of the moment: Perplexity AI ignores robot.txt files and crawls websites even when the site owner says no. https://rknight.me/blog/perplexity-ai-is-lying-about-its-user-agent/
Woah, that is not cool, at all. Even if I don’t care too much for AI bots to crawl and ingest my content, I would expect them to respect those author and site owners who decides otherwise. It’s not the best way to build trust.