Google Adsense College

Matt Cutts of the person that publish, software engineer
Reprint fromAdministrator of website of cereal song Chinese rich guest

Textual
How To Verify Googlebot
Publish at: On September 20, 2006, zhou San, in the morning 11 when 45 minutes

I am heard recently a fewClever PersonageRequirement search engine offers a kind of method to come test and verify robot of a capture is authentic. After all, any rubbish makers can name their capture the robot with Googlebot, claim oneself are Gu Ge. So, you should trust what capture robot, should tackle again what?

We hear the IP address list that the globallest demand is a Googlebot to announce everybody. The problem of this practice is, if / when the IP address limits of the capture tool that becomes us is changed, be not everybody to know to check. In fact, creeping group has removed a few years ago the IP address of Googlebot, a when they encounter real trouble is the net canal in the program that reminds the IP of a few Googlebot limits is written in them people. So the members of creeping group offerred another kind of method to come Googlebot of test and verify. Here is a solution that creeping team members offer (agree via them here quote) :

Tell a website the manager please people, best method looks is use domain name analytic server (DNS) will check every case. The technique of test and verify that I recommend is to do retrorse DNS to search, checking this name is to be inside Googlebot.com domain name, use this Googlebot.com name to do next corresponding to DNS->IP search; for example:
(translator notes: It is Linux command reachs executive result below)

>Host 66.249.66.1
1.66.249.66.in-addr.arpa Domain Name Pointer crawl-66-249-66-1.googlebot.com.
(Crawl-66-249-66-1.googlebot.com of finger of 1.66.249.66.in-addr.arpa domain name)

>Host Crawl-66-249-66-1.googlebot.comCrawl-66-249-66-1.googlebot.com Has Address 66.249.66.1
(the IP address of Crawl-66-249-66-1.googlebot.com is 66.249.66.1)

It is insufficient that I think to do retrorse DNS to search only, because maker of a rubbish can build retrorse DNS,will point to Crawl-a-b-c-d.googlebot.com.

Also our in-house technical help center offers me this solution, so I think this is the official method of Googlebot of a test and verify. For from ” of the government ” the capture inside Googlebot IP limits, capture robot should respect tradition of Robots.txt and we in-house lead plane bear, make Gu Ge divides crawl nevertheless thereby your website.

(thank N. With J. For the help that this article provides, the thing that they introduced to crawl the respect is involved) .

Tags: , Cutts, engineerReprint, fromAdministrator, Matt, person, publish, software