List of Googlebot IPs
Sep 11

List of Googlebot IPs

google.txt: I don’t know how well this is kept up-to-date, but it purports to be a list of all the IPs the Googlebot comes from.

This is of interest after the last post, because Google says you can detect the Googlebot User Agent and bypass login pages to index content behind subscription walls. But it also means that anyone can bypass your login page as well by changing their User Agent.

But, if this is an accurate list of Googlebot IPs, then you could detect both the User Agent and the IP. The trick, of course, is to make sure you have a an up-to-date list of IPs. I imagine it changes a lot, but of the 281 lines in this file, several are Class C subnets, which would encompass ~250 other IPs, so the actual number of potential IPs in this list is into the thousands.

by Deane   September 11, 2008 9:27 AM   |
  tags: 
Google

Comments

by Sennafan,   September 12, 2008 10:49 AM  

The folks over at IP delivery (John Heard), they are probably the most aggressive in keeping track of all major search engine spider IP addresses. They use the IP addresses always save for their IP delivery software which is pretty potent and powerful stuff.

They have a subscription-based service where you can receive immediate alerts when new IP addresses from search engines (such as Google) are detected.

Add Comment

Want to advertise on this site? Contact FM.
Laser Toner Cartridges UK laser toner, toner cartridges, hp toner, lexmark toner, samsung toner, canon, toner, epson toner, oki toner, kyocera toner, xerox toner, remanufactured toner, compatible toner
Direct TV Deals Free 4 room direct tv deals. no equipment to buy. free fast professional direct tv installation. this is the best direct tv deal available anywhere.
SEO Article Learn from the experts with our SEO article.
rope light Shopping with birddog distributing, inc., gives you access to the lowest prices, the best customer service and the quickest delivery times possible.
Laptop AC Adapter We offer genuine factory direct replacement AC adapters.
Direct TV Best satellite TV deals.
Direct TV Deals Direct TV programming deals are varied and include packages containing from 50 channels up to over 250 channels.
8mm film to DVD Retain family memories with the only frame by frame digital restoration service in the United States for your 8mm film to DVD today
Rubber Stamp Shop for custom self-inking stamps, hand stamps, address stamps, label stamps, check endorsement stamps, check deposit stamps, date stamps, pre inks, pocket stamps, ink and much more!