Search Engine Robots - How They Work, What They Do (part Ii)
|
|
If your site isn't found in the search engines, it is probably because the robots couldn't deal with it. It could be something as simple as not being able to find the site, or it may be more complicated issues involving the robot's not being able to crawl the site or figure out what your pages are all about.
Submitting your site to the major search engines: that will help with the "can't find it" problem. Even having links pointing back to your site can be enough to attract the search engine robots. Google, for example, suggests that you may not have to submit your pages; they will find your site if you have a link pointing back to it from at least one other site on the web.
If the robots can find your site but can't make sense of it, then you may need to look at the content and technology used on your pages. Frames, Flash, dynamically generated pages, and invalid HTML source code can cause problems when the search engine robot tries to access your web pages. While some search engines are beginning to be able to index dynamically generated pages and Flash (e.g. Google and AllTheWeb), use of some of these technologies can hinder your ability to be indexed by the search engine robots.
Text in images cannot be read by the search engine robots. Using ALT image text is an important way to help the robots "read" your images. Websites with extensive images rely heavily on ALT text to present their content.How Do I Get The Most Out Of Indexing?
If you know what to "feed" the spidering robots you will help yourself with search engine ranking.
Having a website full of good content is the major factor. Search engines exist to serve their visitors, not to rank your website. You need to be sure to present yourself in your site in the way that will be most useful to the search engine visitor. Each search engine has its own idea of what is important in a page, but they all value text highly. Making sure that the text on your pages includes your most important keyword phrases will help the search engine evaluate the content of those pages.
Making sure that you have good title and meta tags will further assist the search engines in understanding what your page is about. If the text on the page is about widgets, the title is about widgets, and the meta tags are about widgets, the search engine will have a pretty good idea that you are all about widgets. When their visitors search for widgets, the search engines know to list your site in the results.
A sitemap page is a very good way of giving the search engine robot every opportunity to reach your website pages. Since robots click through the links of your web pages, make sure that at least your most important pages are included in the sitemap; you may even want to include all your pages there, depending on the size of your site. Be sure to add a link to the sitemap page from each page on your site.
Another important consideration is that of keeping all of your pages within a small number of "clicks" from your top page. Many robots will not follow links more than two or three levels deep, so if your "widgets" page can only be reached from your home page by following multiple links (e.g. home page >> about us page >> products page >> widgets page), the robot may not crawl deep enough to get to the widgets page.Testing Your Website For Search Engine Robot Accessibility
To get an idea just what the search engine robot "sees" on your page, you can look at the Sim Spider tool. You may be surprised at how different your site looks to the robot. You can find this tool at http://www.searchengineworld.com/cgi-bin/sim_spider.cgi
You will see text and ALT image text show up in the results. If your entire website is built in Flash, you will see nothing at all because robots don't understand Flash movies.The Bottom Line
When it comes to search engine robots, think simply. Lots of good content and text, hyperlinks the robots can follow, optimization of your pages, topical links pointing back to your site and a sitemap will help insure the best results when the robots come visiting.
Resources
*SpiderSpotting - Search Engine Watchhttp://searchenginewatch.com/webmasters/spiders.html
*Robotstxt.orgList of robots and protocols for setting up a robots.txt file. http://www.robotstxt.org/
*Spider-FoodTutorials, forums and articles about Search Engine spiders and Search Engine Marketing. http://spider-food.net/
*Spiderhunter.comArticles and resources about tracking Search Engine spiders. http://www.spiderhunter.com/
*Sim Spider Search Engine Robot SimulatorSearch Engine World has a spider that simulates what the Search Engine robots read from your website. http://www.searchengineworld.com/cgi-bin/sim_spider.cgi
Daria Goetsch is the founder and Search Engine Marketing Consultant for Search Innovation Marketing, a Search Engine Optimization company serving small businesses. She has specialized in Search Engine Promotion since 1998, including three years as the Search Engine Specialist for O'Reilly Media, Inc., a technical book publishing company.
Copyright © 2002-2005 Search Innovation Marketing. http://www.searchinnovation.com All Rights Reserved.
Permission to reprint this article is granted if the article is reproduced in its entirety, without editing, including the bio information. Please include a hyperlink to http://www.searchinnovation.com when using this article in newsletters or online.
|
|
|
Created & Maintained by Empower! CMS Web Sites
Host2Sell Web Hosting | Emarketing Workshops | Site SEO Review | FREE NewsletterUse Search Engines For A Guaranteed Web Site Promotion
For your web site to succeed, you must use is search engines optimization. Web sites definitely need top rankings in major search engines such as Google, Yahoo!, AOL, and MSN. The higher the ranking, the more likely viewers will come and visit your web site. Your site should stay in one of the top positions in the search engines to draw the largest amount of customers.Guaranteed web site promotion needs you to use keywords strongly within the text of your web site. Keywords help not only viewers find your site but the web crawlers the search engines also. ...(related: Search Engine Optimization)
Get A Top Position In Google In Three Days!
So how do you get a top listing in Google and how do you get into Google quickly.Lets begin with the latter. You can submit your site using Googles submit site tool but amazingly enough this is NOT the fastest way to get listed.Here's the secret to getting listed in Google like lightening PRESS RELEASE'SYep issue a press release relating to your website you will see your site in all the major search engines in 3 or 4 days. Here's a link to one of my press releases http://pressreleases.newspad.com/pr/20054/pr224817.htmlTo issue a press release go to http://prweb.com It's free to do.Ok now that your sites in Google how are we going to get a top spot for keywords related to our site. Well I'...(related: Search Engine Optimization)
The Power Of Topic Specific Search
What are Topical Search Engines?Simply put, topical search engines are search engines focused on a specific industry, se...(related: Search Engine Optimization)
Tread Towards A Successful ?internet Research?
Internet is a terrific resource containing billions of web pages dedicated to thousands of topics. Since the amount of information available on the Internet is so vast and mind baffling you may feel lost.Your expectations from "The Information Superhighway" will crash if you proceed with the general view that exaggerates the ease of Internet usage. What is required is a moderate, balanced approach keeping one's head on shoulders. Approaching the net should be similar to any other research endeavor i.e. by adopting a formal strategy to maximize results. "Motivation" is the key word here. If you do not have a serious research goal, you cannot dig far.To make the search a more meaningful exer...(related: Search Engine Optimization)
Yahoo Dopey, Msn Goofy, Google Is Mickey Mouse Lost In A Sandbox
Seventy-two days ago Googlebot first showed up and crawled over 250 pages of a brand new domain in an experiment that has had an odd cartoonish character to it, where unexplained things happen with sometimes dark foreboding, a kind of Fantasia online.If you're unfamiliar with the Disney animation classic, Fantasia, Mickey Mouse plays a sorcerers' apprentice who wreaks havoc oneevening as he dons the bosses magic wand and merrily destroys the castle. Comparing Google to Mickey Mouse is probably not often taken to mean...(related: Search Engine Optimization)
Expert Help From Google Answers
Web users turn to search engines for answers to their questions. This is usually done through various levels of searching the engine's database. Sometimes though, no matter how hard they try, searchers can't find the information they need. Maybe they're not familiar with how to narrow and focus searches, or they may not have enough background in the subject they're researching to recognize the answer they need. Google Answers offers a solution.With Google Answers, you hire an experienced researcher or specialist in your field of inquiry to provide answers to your questions. In a sense, this process works much like bidding on an...(related: Search Engine Optimization)
Microsofts New Search Engine
I tried out Microsoft's new search engine (beta version) the other day. They have been working on it quite secretly now for almost a year. It has been rumored to be a Google killer once in full public release. They say its accuracy and ability to return relevant results will far surpass any other search engine on the Internet. That remains to be seen. I tried various combinations of single, double, and triple word/phrased searches to inspect the results.To the casual observer...(related: Search Engine Optimization)
Seo Tips - Google Has Changed - Learn Why And What To Do
Google now checks the year your domain name was first registered.This just makes sense. Those that care about their domain and their brand will register their domain for a long time. This demonstrates commitment. Of course it's not very hard for a spammer to do the same but the upfront costs are just that little bit higher and might help act as a deterrent. If anything a short ...(related: Search Engine Optimization)
The Sandbox Effect
What once many people thought they had a penalty, is now being called the Sandbox Effect and is causing new web sites not to rank very well in the search results of Google, not even for the least competitive phrases. Meaning that a filter is being placed on new web sites and cannot rank very high for most words or phrases for a certain amount of time.This does not mean it's punishment for anything the webmaster did with their site, such as using Spam or anything like that. The probation likely don't apply to the web site, but instead to backlinks. After the link stays...(related: Search Engine Optimization)
Absolute Top Five Search Engine Marketing Myths Uncovered!
It's no secret that Search Engine marketing can drive significant amounts of very qualified traffic to a web site ? as 85% of Internet users utilize search engines to find/research for goods and services. The problem for many companies is the difficulty they face sifting through conflicting information and hyperbole! Here is my top five list of myths that need to be run to ground.Big Picture Myth One ? Search engine ranking leads are not as good as those which originate from other forms of traditional marketing (print, direct mail, PR, etc.) ? t...(related: Search Engine Optimization)
site-map - Copyright © 2006 Empower! Web Design | All Rights Reserved. | Search Engine Optimization