# AI.txt - AI Training and Usage Policy for gtmengine.ai # This file specifies how AI systems may access and use content from this website. # Last updated: 2024 # Specification: https://site-url.com/ai.txt # ============================================================================= # GENERAL POLICY # ============================================================================= # By default, we allow AI systems to access our public content for training # and inference purposes. However, individual pages may opt out using the # noindex robots meta tag. # # If a page includes , AI systems # should respect that directive and not use that content for training. # ============================================================================= # Default Policy: Allow all AI bots User-agent: * Allow: / # ============================================================================= # BLOCKED AREAS (Always excluded from AI training) # ============================================================================= # These areas contain internal tools, admin interfaces, or dynamic content # that should never be used for AI training purposes. Disallow: /studio/ Disallow: /api/ Disallow: /_next/ Disallow: /admin/ # ============================================================================= # SPECIFIC AI BOTS # ============================================================================= # We explicitly allow the following AI systems to access our content. # Individual pages can still opt out using noindex meta tags. # OpenAI (ChatGPT, GPT models) User-agent: GPTBot Allow: / # OpenAI Search Bot User-agent: OAI-SearchBot Allow: / # Google AI (Bard, Gemini, Search Generative Experience) User-agent: Google-Extended Allow: / # Anthropic (Claude) User-agent: anthropic-ai Allow: / User-agent: Claude-Web Allow: / # Common Crawl (used by many AI companies for training data) User-agent: CCBot Allow: / # Perplexity AI User-agent: PerplexityBot Allow: / # Cohere AI User-agent: cohere-ai Allow: / # Meta/Facebook AI User-agent: FacebookBot Allow: / User-agent: Meta-ExternalAgent Allow: / # Apple Intelligence User-agent: Applebot-Extended Allow: / # Diffbot (AI-powered web data extraction) User-agent: Diffbot Allow: / # You.com AI Search User-agent: YouBot Allow: / # ByteDance (TikTok) AI User-agent: Bytespider Allow: / # ImagesiftBot (image AI) User-agent: ImagesiftBot Allow: / # ============================================================================= # USAGE POLICY # ============================================================================= # We allow both commercial and non-commercial use of our public content # for AI training purposes, subject to the following conditions: # # 1. Respect noindex directives on individual pages # 2. Do not access content behind authentication or paywalls # 3. Respect rate limiting and crawl delays specified in robots.txt # 4. Attribution is appreciated but not required # 5. Do not use content for harmful, illegal, or unethical purposes # ============================================================================= # Commercial use policy Commercial-use: allowed # Training use policy Training-use: allowed # ============================================================================= # CONTACT # ============================================================================= # For questions about our AI policy or to request removal of specific content # from training datasets, please contact: https://gtmengine.ai/company/contact # =============================================================================