# robots.txt for [Your New Project] # https://aiplex.com # Allow all crawlers full access User-agent: * Allow: / Disallow: /privacy-policy Disallow: /terms-condition Disallow: /cookies # Specific crawling instructions User-agent: Googlebot Allow: / Crawl-delay: 1 Disallow: /privacy-policy Disallow: /terms-condition Disallow: /cookies User-agent: Googlebot-Image Allow: / Disallow: /privacy-policy Disallow: /terms-condition Disallow: /cookies User-agent: Bingbot Allow: / Disallow: /privacy-policy Disallow: /terms-condition Disallow: /cookies Crawl-delay: 1 User-agent: Slurp Allow: / Disallow: /privacy-policy Disallow: /terms-condition Disallow: /cookies Crawl-delay: 2 # Sitemap location Sitemap: https://aiplex.com/sitemap.xml