Effectively Using Robots Meta Tags
|
|
The "robots" meta tag, when used properly, will tell the search engine spiders whether or not to index and follow a particular page. For the purposes of this article, we will be using the "( )" symbols to represent the "< >" in html coding.
Some examples of robot usage are as follows:
(meta name="robots" content="index,follow")
(meta name="robots" content="noindex,follow")
(meta name="robots" content="index,nofollow")
(meta name="robots" content="noindex,nofollow")
Let us first examine what these terms mean before we explain the usage for each one:
"index"- This directive tells the search engine robots (or spiders) that it is okay to index the page. Another words, you are allowing the search engine to include your page within their search directory.
"noindex"- Using this tag, you are informing the robots that this page should not be indexed. Simply put, this page will not appear in their search directory.
"follow"- When you use this tag, you are telling the search engines that you want their robot to follow any links that are found on that page.
"nofollow"- The opposite of the above definition, this directive will tell the robots not to follow any links on your page.
Putting It All Together:
With the robots tags explained, let's examine the usage for each one.
1. (meta name="robots" content="index,follow")
This tag will be used when you want the search engine spiders to index the page and follow the links to other pages. Most search engines use this setting as a "default" setting. It is possible that you may not even need to use this tag if you want the search engines to follow and index the page. However, an article at Search Engine World (searchengineworld.com/metatag/robots.htm) suggests that Inktomi does not use this as their default setting. Instead, they use the "index, nofollow" tag.
Better safe than sorry!
There has been much debate over whether or not it is necessary to use this tag. If there is even a slight possibility that some search engines do not use this as the default setting, then it would only make sense to include this tag if you want your page included in their search directory AND your links to be followed. Do the research and decide for yourself.
2. (meta name="robots" content="noindex,follow")
This tag can be used to tell the search engines that you do not want the page included in their directory, but you DO want them to follow the links that lead to other pages. A good example of its usage would be your disclaimer or privacy policy pages. You may not want these pages to show up in the search engines if they are only important to your actual visitors. However, if the links on these pages point to other pages that you want the search engines to find, then you would still want the spiders to "follow" those links.
3. (meta name="robots" content="index,nofollow")
This tag will allow your page to be indexed in the search engines, but any links on that page will not be followed.
4. (meta name="robots" content="noindex,nofollow")
When using this tag, the search engine spiders will not include this page in their directory and will not follow any links on the page either.
Where does the "robots" tag belong?
The "robots" meta tag should be used within the (head) and (/head) tags of your page. These tags are located at the top of the html coding. It will look something like this:
(html)
(head)
(title)Title of your page goes here(/title)
(meta name="keywords" content="word1,word2,word3,word4")
(meta name="description" content="A brief description of the content of this page.")
(meta name="robots" content="index,follow")
(/head)
(body)
Your webpage information here.
(/body)
(/html)
More Robots Tags
Google automatically archives a page as it crawls it. This is called a "cached" version of the page. Visitors can retrieve the archived version of the page by clicking on the "cached" link within Google's search results. If you do not want your content to be archived, you can use the following tag:
(meta name="robots" content="noarchive")
*This will only prevent your page from being "cached". If you do not want your page to be indexed at all, you will still need to include the "noindex" tag.
Another alternative to the above tag is the tag that specifically addresses Google only. If you want other search engine robots to archive your site, but you would like to prevent Google from doing so, then you can use the following tag:
(meta name="googlebot" content="noarchive")
The Misuse of Robots Tags
Something that has been popping up on websites everywhere is the Google indexing tag. This is a silly little tag that is not necessary. Some people think this tag helps Google to spider your site, but this simply isn't true. The tag looks like this: (meta name="googlebot" content="index,follow"). Some website owners believe that by specifying "googlebot" that their site has the advantage of being spidered faster and listed by Google. According to Google's web crawler information, you may use the noindex, nofollow, or noarchive tags when you DO NOT want Google to cache, index, or follow that page. Google's default setting is to index and follow the links on the page, so this "so called" googlebot index/follow tag that some site owner made up one day is completely unnecessary.
Another silly little tag--- The "Revisit-After" Tag
(meta name="revisit-after" content="90 days")
(meta name="revisit-after" content="15 days")
I'm not sure where this myth was started. Today, you will find this tag all over the Internet. Webmasters have even promoted it, claiming that it actually works. Are we so naive to believe the search engine spiders need to know when to come back? I have never used this tag, and my site has no problem with being crawled on a regular basis. Even some SEO (search engine optimization) sites are claiming its value. This comes back to the importance of researching the topic thoroughly first. My research came from some interesting information on WebmasterWorld.
It is important to examine the correct usage of the "robots" tag before applying it to your website. Incorrect usage of tags could result in spidering errors that cause robots to completely ignore the page all together. If you are interested in learning more about web robots, this great little site will provide you with the information you need to use them effectively: www.robotstxt.org/wc/robots.html
Stephani Richardson is a work at home mother of 4 who has been actively involved with affiliate marketing and home business opportunities since December 1999. She owns and operates several business related websites including 1 Work At Home Dot Com.
|
|
|
Created & Maintained by Empower! CMS Web Sites
Host2Sell Web Hosting | Emarketing Workshops | Site SEO Review | FREE NewsletterTraffic For Webmasters
"If you build it, they will come"; is an age old phenomenon for webmasters that they develop the website and visitors themselves would visit that. This may be true for only a handful of websites but the most important and crucial topic for any webmaster today is to how to get targeted traffic to his website.How good is a website without traffic, only webmasters know that very well and this is no doubt the biggest question of the new Internet age. The new webmasters have to face a lot of problems in order to attract the targeted traffic to their website. But by following few most used methods they can attract visitors of their particular niche group in a very safe manner. These methods are used and recommended all over the world and have been best...(related: Web Development)
How To Make Your Website More Successful? (part I)
Building a website and getting it online is easy. Driving visitors to it is the more difficult part. Most people are not patient enough when it comes to build up traffic. They expect thousands of visitors a week after they go live with their website. But that is not how it works. We share some secrets of how to make your website more successful.A) Provide content: Search engines love content. As...(related: Web Development)
Your Website Reflects Your Business
Some left shoes are in isle 5, while the right shoes are in isle 3. Shoe hills are in random places. You can barely walk through the store without stumbling over a shoe.No employees are in sight. As you are desperate for help, you finally make it past the heaping hill of shoes to the back of the store and find a button that says "Page for Help". You press the button and out comes a card that says "please leave your mailing address, we'll send you a letter". Frustrated, you leave the store, and finally realize there is a Footlocker across the street.We all know how first impressions can make or break a potent...(related: Web Development)
Why You Need A Web Site
"I have a small business with just a couple of employees and I don't think my product can really be sold online. Do I need a web site?"In this day and time, there is very little that can't be sold over the Internet. There are over 20 million people shopping online...(related: Web Development)
Study Your Sites Performance Via Hit Tracking
Knowledge is power.And undoubtedly, it is the key to making a substantial fortune online. Consider these: what if you knew the sited your visitors came from before they arrived at yours, what they were checking out before they took a look at your pages, and where they went after visiting your place? What if you knew which of your links were being clicked, how much they were being used, and how well they are performing? What if you knew which among the pages in your site attracts the most attention?Again, knowledge is power and knowing all these things would lead to better results from your site.Thankfully, technology has caught up with this highly particular necessity. A lot of programs...(related: Web Development)
How To Promote Your Law Firm Website On The Internet For Maximum Profit
Making maximum profit from your law firm website is important, but as a lawyer you might not have any clue about search engine optimization and Internet marketing for your legal site. However, it is not so difficult to poise your website to be successful online even if you are not experienced with Internet marketing and driving website traffic. All you have to do is review the following tips and include them in your website marketing plan. Before you know it you will notice that your website counter is recording more hits per day and telling you that your directories, keywords, and links are working.Tip #1 - Submitting to Law DirectoriesTake some time online to orient yourself with the myriad of law directories available. Register your domain name and the type of law you practice with as many law directories as you possibly can. By ...(related: Web Development)
Easy Steps To Website Development And Promotion
One of the most important parts of establishing a new website is website development and promotion. For people to visit your new site, you have to attract them. After spending so much time drawing up your website, you must also think of ways to make customers want ...(related: Web Development)
Improve Web Sales Figures Forever And Ever Amen!
"How do I improve web site sales figures?" The CEO roared at the web developer. The web developer looked at his boss a little confused and mumbled something about it not being his core competence with him being a programmer and all. So next the CEO went to his marketing department and bellowed "How do I improve our web site sales conversion rates?" A few mouths opened as if to speak but nothing came out until finally exasperated the CEO tearing at his hair i...(related: Web Development)
Cost Effective And Powerful Dynamic Websites W. Fantastico Web Hosting
If you are developing your website on a host that charges less that $5 a month, make plenty of backups as you are likely to loose your website files when the webhost goes out of business without notice. Indeed, be extremely wary of any such host. Once your website is attracting getting visitors, get your website hosted with host that does not foolishly try to over sell his server resources. And, expect to pay at least $15 a month or more for the added security and reliability of such a host.I highly recomend you look for a host that has the latest version of cpanel and fantastico installed. Fantastico is one of the greatest tools for web developers that I have seen in a long time. I can remember the time when ins...(related: Web Development)
The Evolution Of Search
There have been many significant changes to the face of search over the last several years with engines becoming more intelligent than ever before. Today's users expect mainly fast, easy, relevant and satisfactory search results. In response to this search engines have responded by giving users more control over search results than ever through the emergence of alternative search engines.One instance of these so-called alternative search engines goes by the name of Nutch (http://www.nutch.org/). Nutch is a two-year-old open source project, which has been hosted previously at Soundforge and backed by a non-profit organization. Since...(related: Web Development)
How Does Your Website Make Me Feel?
When people think about the Internet, they think about technology. When people hear that I am a Website strateg...(related: Web Development)
Website For The Company Store
If your company own one or more stores, you might be wondering if it would be a good idea to have a website. You might also be wondering if a website for the company store would be profitable for your business.Those are very legitimate questions.I worked as a marketing director for a store for some times, and I was always harassed by salesman wanting me to by publicity for every imaginable form of paper advertising. Some offer were interesting, most were not.Are you in the same situation?
How One Word Or Even One Letter Can Boost Conversion Rates By Over 400%!
Recently I was reviewing the keyword specific conversion rate data of a consulting client of mine. I have been working with this client...(related: Web Development)
site-map - Copyright © 2006 Empower! Web Design | All Rights Reserved. | Web Development