User-agent: * Disallow: /search.html?* Disallow: /download/ User-agent: BLEXBot Disallow: / User-agent: proximic Disallow: / User-agent: GrapeshotCrawler Disallow: / User-agent: MJ12bot Disallow: / User-agent: linkdexbot Disallow: / User-agent: AhrefsBot Disallow: /