Webservers Software
|
|
With Robots.txt Editor you create robot exclusion files by selecting all robots or a specific user-agent and adding documents and/or directories by entering the path names manually or by selecting them using FTP.With Log Analyzer you can get the full statistics of robots visiting the pages of your website. Keywords: robotstxt spiders site search visits spider analyzer html robots-txt log analyzer visually generate industry standard robots txt files analize webservers log |
|