Mark Hobley
ROBOTS META TAG
The Robots META tag is a simple mechanism to indicate to Web Robots if a page should be indexed, or links on the page should be followed.

Note: Not all robots support this tag.

Like any META tag it should be placed in the HEAD section of an HTML page:

<html>
<head>
<meta name="robots" content="noindex,nofollow">
</head>
<body>
...

What to put into the Robots META tag

The directives should be separated by commas.

The current defined directives are:

index
noindex
follow
nofollow
all
none

all is the equivalent of index, follow
none is the equivalent of noindex, nofollow

Some examples:

<meta name="robots" content="index,follow">
<meta name="robots" content="noindex,follow">
<meta name="robots" content="index,nofollow">
<meta name="robots" content="noindex,nofollow">

Note the "robots" name of the tag and the content are case insensitive.

Do not specify conflicting or repeating directives such as:

<meta name="robots" content="index,noindex,nofollow,follow,follow">