There may be times when you will want to (try) control the behavior of search engine robots. You may want to tell them not to index (store your pages on the Search Engine Servers) one of your pages or not to follow a link on a certain page. This is where creating a meta robots tag comes in.
In the past, meta robot tags were used more often. Now more can be accomplished through the use of robots.txt.file. Whether or not you have access to create and add a robots.txt file depends on your web hosting.
If you are unable to create a robots.txt.file, using the meta robots tag is the only method available to control a limited amount of the behavior of the search-engine spiders.
There are four directives available for the search engines to follow: index, noindex, follow, and nofollow. Use only one of these four at a time:
- Index: including index in the meta robots tag tells the spiders to index that page.
- Noindex: including noindex in the meta robots tag tells the spiders not to index that page.
- Follow: including follow tells the spiders to follow links on that page.
- Nofollow: including nofollow tells the spiders not to follow links on that page.
You can also use CONTENT= “all” instead of “index,follow” or CONTENT= “none” instead of “noindex,nofollow”.
Depending on what you are trying to accomplish, you may want to invest in a web hosting service that provides the use of robots.txt. file. It provides another layer of protection.
The meta robots tag is found in the header of an HTML document, and its syntax is as follows:
<HEAD>
<META NAME= “robots” CONTENT= “index,follow”>
Or
<META NAME= “robots” CONTENT= “noindex,follow”>
Or
<META NAME= “robots” CONTENT= “index,nofollow”
Or
<META NAME= “robots” CONTENT= “noindex,nofollow”>
</HEAD>
Steps to Creating a Meta Robots Tag:
1.Find a web page on your site that you do not want the search engines to index.
2.Open your text editor or web development program and find the <HEAD> tag.
3.Between your <HEAD> and </HEAD> tag, type <META NAME=
“robots” CONTENT= “noindex,nofollow”>.
Use this code if you do not want the search engines indexing a page
and following any links from that page.
4. Save the file and upload to your web server.
