robots | VV's Corner

my collection of random thoughts, how-to's and code snippets

About

Currently viewing the category: "robots"

robots.txt

By admin On April 6, 2007 · Add Comment

A robots.txt file in your web site’s root directory informs robots who follow the Robots Exclusion Protocol on which areas of your site can be indexed and which areas cannot be indexed. A simple robots.txt implementation tells all robots that your entire site is indexable: User-agent: * Disallow: But be careful not to do the [...]

[Continue Reading...]

Recent Posts
Tags
adwords analytics api auto-tagging autotagging earth gclid geo google google maps api static google webmaster tools 404 widget list mapplets maps python regex regular expressions set street streetview string view

About