User-agent: * Disallow: / User-agent: Googlebot Allow: / Disallow: /billing Disallow: /bookmark-export Disallow: /illustrations Disallow: /diagrams Disallow: /illustration_images Disallow: /login Disallow: /msearch Disallow: /payment Disallow: /account-creation-confirmation Disallow: /profile Disallow: /project/ Disallow: /projects Disallow: /search Disallow: /sign-up Disallow: /users Disallow: /viewer-export/ Disallow: /get-copilot Disallow: /get-copilot-ca Disallow: /get-copilot-nyc # Yahoo Search should follow Bing rules User-agent: Bingbot Allow: / Disallow: /billing Disallow: /bookmark-export Disallow: /illustrations Disallow: /diagrams Disallow: /illustration_images Disallow: /login Disallow: /msearch Disallow: /payment Disallow: /account-creation-confirmation Disallow: /profile Disallow: /project/ Disallow: /projects Disallow: /search Disallow: /sign-up Disallow: /users Disallow: /viewer-export/ Disallow: /get-copilot Disallow: /get-copilot-ca Disallow: /get-copilot-nyc User-agent: DuckDuckBot Allow: / Disallow: /billing Disallow: /bookmark-export Disallow: /illustrations Disallow: /diagrams Disallow: /illustration_images Disallow: /login Disallow: /msearch Disallow: /payment Disallow: /account-creation-confirmation Disallow: /profile Disallow: /project/ Disallow: /projects Disallow: /search Disallow: /sign-up Disallow: /users Disallow: /viewer-export/ Disallow: /get-copilot Disallow: /get-copilot-ca Disallow: /get-copilot-nyc # Recommended set of disallows from here: # https://www.coywolf.news/seo/google-announces-method-for-sites-to-opt-out-of-llm-training/ # Technically not required given the above blanket disallow, but keeping to be explicit User-agent: CCBot Disallow: / User-agent: ChatGPT-User Disallow: / User-Agent: FacebookBot Disallow: / User-agent: GPTBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: Omgilibot Disallow: / ###################################################### # And some extra ones for good measure User-agent: Amazonbot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: AwarioRssBot User-agent: AwarioSmartBot Disallow: / User-agent: Bytespider Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-Web Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: facebookexternalhit Allow: /*?*smid= User-agent: magpie-crawler Disallow: / User-agent: omgili Disallow: / User-agent: peer39_crawler User-agent: peer39_crawler/1.0 Disallow: / User-agent: PerplexityBot Disallow: / User-agent: Twitterbot Allow: /*?*smid= User-agent: AhrefsSiteAudit Allow: / User-agent: AhrefsBot Allow: / Sitemap: https://up.codes/sitemap.xml