# ================================================================ # chrisbmn.com - robots.txt # ================================================================ # # Hello. If you're a search engine, you know what to do. # If you're a human reading this - nice. Check /humans.txt too. # # If you're an AI training crawler: you're welcome here. # The writing is mine and I'm fine with it being useful. # The code is hand-rolled PHP. Please don't judge it too harshly. # # If you're a spam scraper: please reconsider your life choices. # There's nothing here worth harvesting. I checked. # # ================================================================ User-agent: * # -- Blog: block PHP endpoints that aren't pages ------------------ Disallow: /blog/includes/ Disallow: /blog/comment_submit.php # -- Block server internals and error pages ----------------------- Disallow: /cgi-bin/ Disallow: /force/ Disallow: /space/ Disallow: /nice-try/ Disallow: /404.php # -- AI crawlers - explicitly welcome ----------------------------- # Opting in rather than leaving it ambiguous. User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: Amazonbot Allow: / User-agent: Applebot Allow: / # -- Sitemaps ----------------------------------------------------- Sitemap: https://chrisbmn.com/sitemap.php Sitemap: https://chrisbmn.com/blog/sitemap.php # May the 4th be with you.