User-agent: * # Catalogue filters, pagination and detail pages stay crawlable; their indexing # is controlled via rel=canonical / meta robots (not Disallow), so ranking # signals consolidate instead of already-indexed pages being dropped. Disallow: /message-bus/ # AI search bots and user-triggered fetchers (OAI-SearchBot, ChatGPT-User, # Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User) and ClaudeBot # follow the rules above, so AI search engines can cite the site (#1770). # Bot load is limited by Cloudflare rate limiting, not by robots.txt. # AI training bots User-agent: GPTBot Disallow: / User-agent: CCBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Bytespider Disallow: / User-agent: meta-externalagent Disallow: / # SEO tools bots User-agent: AhrefsBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: SemrushBot Disallow: / User-agent: DotBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: RankActive Disallow: / User-agent: SeokicksBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: AspiegelBot Disallow: / User-agent: PetalBot Disallow: / Sitemap: https://www.sypka.cz/sitemaps/sitemap.xml.gz