Category Archives: SEO

robots.txt

Posted by admin on April 6, 2007 0 comments

A robots.txt file in your web site’s root directory informs robots who follow the Robots Exclusion Protocol on which areas of your site can be indexed and which areas cannot be indexed. A simple robots.txt implementation tells all robots that your entire site is indexable: User-agent: * Disallow: But be careful not to do the [...]