Release From Google Sandbox Only To Search The Playground
|
|
The Google Sandbox Effect has been discussed at length in ourcase study of a new website first crawled in May by Googlebot.We can now further the case study with indexing comparisonsand discuss interesting Googlebot crawler behavior afterrelease, at the 75 day mark, of the study website from thatvery confining Sandbox.
This case study is not for the faint of heart - those justlaunching a new web business on a new domain name with hopesof instant indexing and immediate traffic may find theirwebsite very lonely for two and a half months - if it is in acompetitive market segment. You may as well plan to stay inthe Google Sandbox for at least 45 days on average. If someearly release stories are to be believed, search phrasesnobody wants to play with are taken pity on by Google and senthome for early release.
Those non-competitive or obscure search phrases seem to beseen as good, quiet little children, playing by themselves inSandbox playground and are sent home early on good behavior.Googlebot probably sees good behavior as playing well withothers, like a good little baby domain and NOT beingcompetitive as some young domains can be. Throwing sand inother childrens' faces and insisting on having your siteindexed, throwing sand out of the Sandbox with your brightplastic toy shovel and bucket will not be allowed.
Now that the site discussed in this study is out of theSandbox, it still lingers on the playground, unable to escapethe community park and leave for the business world to playwith the big boys in the outside world. It does indeed taketime to grow up and be the model citizen in this new searchplayground. Though on the first full day after this first weekof being released from the sandbox, the site has gotten 68visitors referred by searches done at Google, the firstreferred search traffic coming into the site. MSN has sent 8visitors, Yahoo has sent 6, 4 came from AOL searches, 2 fromNetscape and 1 from Dogpile.
The indexing behavior of Yahoo and MSN has been nothing shortof bizarre with numbers of indexed pages increasing rapidlyover the first two months to reflect 6,941 pages indexed until8 weeks into this study and we outlined previously how numberschanged as you click through results pages first upward, thendownward to about half the total of highest numbers listedalong the top of the results pages.
It appears that Yahoo and MSN are playing on the 'slipperyslide' in this playground, climbing to the top of the ladderof results at about 10 week mark showing 8,210 and 6,941 pagesrespectively indexed, then sliding down again to 3,510 forYahoo and 373 for MSN, as of this writing two weeks later onAugust 6. Still, Yahoo will show you only 1,000 (100 pages) ofthose results and MSN will show you only 250 results, or 25pages, no matter how many they claim to index. MSNbot iscrawling the site faster and more consistently than any of theengines, yet shows by far fewer pages indexed than the others.
One of the interesting comparisons between Google and MSN inour Sandbox study is that Google will show you most of whatthey claim to have indexed after you click that link at thebottom of the first page showing only 3 or 4 results when youuse the "site:Publish101.com" query operator then go to thebottom of the page and click the link under the line reading,"In order to show you the most relevant results, we haveomitted some entries very similar to the 3 already displayed.If you like, you can repeat the search with the omittedresults included."
Go ahead and click that link, then you'll be presented withthe claimed total of indexed pages. That number has verysteadily increased since Sandbox release after 75 days fromfirst crawling of this Sandbox study site. The timing andnumbers of indexed pages at Google goes upward, and ONLYupward with VERY distinct patterns noted from raw log files.Crawling schedules seem to have been established for this siteby Google and indexing changes occur on a very regularschedule.
The first observation of Sandbox release was at noon onThursday July 28, seventy-five days from first crawling byGooglebot when a search turned up 379 pages indexed with a"site:Publish101.com" query. That number increased later thesame evening to 3,660 pages at a search done around the dinnerhour Pacific time. Oddly, the next day, Friday July 29, thenumber took a slight hop upward to 3,700 pages and on thefollowing Monday, showed 3,770 pages indexed.
That schedule and pattern have repeated on the second week ofSandbox release when a "site:Publish101.com" query produced5,660 results from from Google for the site on Thursday August4 at just after noon and then nearly doubled at around thedinner hour to 10,700 pages on that same query. A final checkjust now on Saturday shows it at 12,100 pages indexed byGoogle. It should be pointed out to those who wonder about thetotal number of pages that this is a dynamic site with a verylarge archive of articles that increases daily as newsubmissions are contributed by member authors at the site.
Those articles are added through a content management systemon a daily basis by an editor who reviews submissions andprocesses them for approvals or rejections. Those approved aremade live from the home page nightly. We've started doing thison the crawler's schedules as we've noted very regular visitsby Yahoo's Slurp crawler to the site home page just once dailyat around 5pm each evening and Googlebot visiting the homepage only once, at near 11pm nightly, so we've instituted amidnight activation of each day's new article submissions onthe home page of the site so that none of the new pages aremissed by those crawlers. MSNbot seems to hit the home pagemultiple times through the day, so timing is less importantfor MSN.
Crawler activity has been heated, with Yahoo crawling theleast and the slowest, barely seeming to attempt any updatesand the total of indexed pages has not changed for over threeweeks since it peaked at 8,210 pages indexed and then droppedto it's current level of 3,510. As previously stated, Slurpseems to be unhindered by any form of consistency in indexingor crawling behavior. MSNbot has crawled extensively andfairly regularly for weeks, but that odd indexing behavior isa serious flaw in their utility as a search tool.
It should be mentioned here that AskJeeves had been noted tocrawl the site extensively early in this case study anddisplayed a very regular and consistent crawl, but stoppedabruptly three weeks ago on july 13, after hitting most of thepages then available on the site. Teoma, their spider, hasbeen absent ever since and they have not indexed this domainat all since first crawling on May 23, over 10 weeks ago.Clearly, Teoma appears to have the longest Sandbox of all thesearch engines.
Much has been learned in this Sandbox case study about crawlerbehavior, indexing delays, robots.txt requirements and indexupdates at each of the top three search engines. Where thatknowledge leads will, of course, change as algorithms andcrawling schedules are adjusted by MSN, Yahoo and Google. Butvaluable information has been shared that may help otherwebmasters to better understand each of the factors thatdetermine the success of any website.
"Further findings in follow-up articles at the 3, 6 and 9month marks, explore search referrals gained as Google addsmore pages and rankings fluctuations begin to level.Meanwhile, we'd like to encourage others to publicly review their crawler traffic through logs to compare behavior on newdomains to verify findings and disclose indexing behavior and timing for new domains and further document SE indexing as well as crawling behavior.
Copyright © August 6, 2005
Previous Sandbox Case Study Articles:
http://Publish101.com/Sandbox2 http://Publish101.com/Sandbox3 http://Publish101.com/Sandbox4
Mike Banks Valentine is a search engine optimizationspecialist
|
|
|
Created & Maintained by Empower! CMS Web Sites
Host2Sell Web Hosting | Emarketing Workshops | Site SEO Review | FREE NewsletterThe Ultimate Free Google Ranking Tool
The first months my website was online, I was constantly checking the search engines to see if my site was listed under the keywords that I was targeting. And always with the same negative results.The truth is that the keywords that you are targeting are often not showing your site the first months at all on the first 200 search listings. OK,if you try to get your site listed for hoooohjgaagga or something like that, it could get you a first place in no time. But who wants to target that keyword?In fact, I get a lot more traffic from keywords that I haven't thought of using as a keyword in the first place. One tool that I have found online can easily show you the keywords that your site is ranking well for.<...(related: Search Engine Optimization)
Yahoo!/overture Site Match: A License To Steal
Unless you've been living in a cave somewhere, I'm sure you've heard by now, Overture now offers the Yahoo! Search Inclusion under its own branded name--Site Match.According to the page info from Overture, submitting your site to individual search engines is expensive and time-consuming. But with Site Match you can reach millions of users by submitting your pages through one program that powers search results for top web portals such as Yahoo!, AltaVista, AlltheWeb and other sites.Summary of Site Match Benefits (according to Overture):
- More exposure for your site--reach more than 75% of active internet users
- A simple, single point of submission to multiple web portals such as Yahoo!, AltaVista and AlltheWeb
- Frequent refresh of your pages--every 48 hours
- Daily reporting...(related: Search Engine Optimization)
Find Best Keywords For Your Site
Keyword optimisation is probably the most important thing that you want to concentrate on with regards to search engine optimisation (SEO). Unfortunately, not many people know this, or do enough to optimise their sites' keywords. Before I knew the importance of keyword optimisation, I used to pluck any keywords that seemed relevant to my site, insert them into my title tag and meta tags, then submitted my site to the search engines. And then wondered why I didn't really get much traffic.Well, now I've learnt something: make sure you don't rush, and you have to do your d...(related: Search Engine Optimization)
8 Essential Seo Techniques
1) Title Tag - The title tag is the most powerful on-site SEO technique you have, so use it creatively! What you place in the title tag should only be one thing, the exact keyword you used for the web page that you are trying to optimize. Every single web page should have it's own title tag.2) ALT Tags - ALT tags were meant to be for text browsers because the images didn't show in text browsers and the ATL tags would tell the visitor what it's about. You should put your main keyword(s) in the ALT tags, but don't over do it because you could get dropped in the r...(related: Search Engine Optimization)
Link Popularity: Improve Your Search Engine Rankings
What is link popularity?Link Popularity is simply the total number of pages that link to your website. Most search engines, including Google, consider that when one page links to another page, it is effectively casting a vote of confidence for the other page. Therefore, the likelihood of you being the best source of information for their searchers is directly linked to the number of votes you have. It is therefore safe to assume that the higher the number of pages linking to your website, the higher you will be ranked on the search engines. Link popularity has become a critical success factor for search engine optimization within popular search engines.H...(related: Search Engine Optimization)
What Is Search Engine Marketing?
It's in our genes, we're driven to seek. We 'hunt' for food, homes, partners, jobs and lately ? with the advent of the Internet ? information.Even in the early years of the Internet, before it evolved to include Web browsers, there were text based search engines. Back then they had funny names like Archie, and Gopher. It was only later, when the graphical browser was invented, that searching became much easier and accessible for the rest of us.The concept behind Search Engine Marketing (SEM) is quite simple: when a consumer or business person searches the Web through either a text box or by clicking through a directory hierarchy, they are in "hunt mode."...(related: Search Engine Optimization)
How To Improve Your Search Engine Positioning And Increase Traffic Today
Every website has times when traffic is higher than others. However, in the downtimes you need to figure out why your traffic is lower and what you can you about it. The following suggestions have been proven to increase website traffic and will be effective in getting you more customers. However, before implementing these tips into your website promotion ...(related: Search Engine Optimization)
7 Search Engine Resources You Should Be Using Now
Ask any business person who's website is at the top of thesearch engines if his/her site is making money, and the answeris almost always "yes".An example is Glenn Canady, the author of "Gorilla Marketing"who employed only one of these strategies, and it made him over$1 million dollars.The fact is, search engines can get you an enormous amount oftraffic, and it's traffic to your sites that's free. However, inorder to ethically and effectively market in the search engines,you need to use strategies that actually work.Below are three different ways to effectively, and ethically,raise your rankings in the search engines. I've include...(related: Search Engine Optimization)
Black Hat Seo And The Sneaky Redirect
Are shades of grey SEO really Black Hat SEO?Black hat SEO is a strategy which gets a web page or entire site banned from a search engine.A shade of grey is when you use a black hat strategy but your site has not been banned yet. Remember the acronym for YET: You're Entitled Too!There are many different opinions on the subject of Search Engine Optimization. Many folks will deliver advice which will work to get you top 10 rankings but what is really the difference between Black Hat SEO and White Hat SEO?There has been many good attempts to define Black Hat SEO. All are relevant and an example can be found at
site-map - Copyright © 2006 Empower! Web Design | All Rights Reserved. | Search Engine Optimization