User-agent: * Disallow: /*? Disallow: /*.js$ Disallow: /*.css$ Disallow: /404/ Disallow: /api/ Disallow: /cgi-bin/ Disallow: /downloader/ Disallow: /install/ Disallow: /private/