# robots.txt for http://www.taropatch.net/ User-Agent: * Disallow: /amzn/ Disallow: /bn/ Disallow: /liane/ Disallow: /mail/ Disallow: /mail2/ Disallow: /shop/ Disallow: /weather/ Disallow: /forum/post.asp Disallow: /forum/pop_printer_friendly.asp ## Disallow these user-agents completely, assuming they pay any heed to robots.txt User-agent: Download Ninja Disallow: / User-agent: Fetch Disallow: / User-agent: grub-client Disallow: / User-agent: HTTrack Disallow: / User-agent: k2spider Disallow: / User-agent: larbin Disallow: / User-agent: libwww Disallow: / User-agent: linko Disallow: / User-agent: Microsoft.URL.Control Disallow: / User-agent: MSIECrawler Disallow: / User-agent: Net Vampire Disallow: / User-agent: NPBot Disallow: / User-agent: Offline Explorer Disallow: / User-Agent: OmniExplorer_Bot Disallow: / User-agent: sitecheck.internetseer.com Disallow: / User-agent: SiteSnagger Disallow: / User-agent: Teleport Disallow: / User-agent: TeleportPro Disallow: / User-agent: TurnitinBot Disallow: / User-agent: Twiceler Disallow: / User-agent: WebCopier Disallow: / User-agent: WebReaper Disallow: / User-agent: WebStripper Disallow: / User-agent: WebZIP Disallow: / User-agent: Xenu Disallow: / User-agent: Zealbot Disallow: / User-agent: ZyBORG Disallow: /