# robots.txt — safedrop apex marketing site # Private app and API paths are disallowed; marketing pages are open to # search engines and AI crawlers. User-agent: * Disallow: /api/ Disallow: /admin-interface Disallow: /s/ Disallow: /form/ Disallow: /auth Disallow: /messages Disallow: /teams Disallow: /settings Disallow: /admin Disallow: /forms Allow: / # AI crawlers — explicitly welcome User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: Google-Extended Allow: / User-agent: PerplexityBot Allow: / User-agent: Applebot-Extended Allow: / User-agent: CCBot Allow: / # Bytespider ignores crawl-delay and is known for aggressive crawling — blocked. User-agent: Bytespider Disallow: / Sitemap: https://safedrop.com/sitemap.xml Sitemap: https://blog.safedrop.com/sitemap_index.xml