# https://www.robotstxt.org/robotstxt.html User-agent: * Disallow: /user/ Disallow: /api/ # Allow everything else Allow: /