| Author |
Message |
NateM
MD user level 5
Joined: June 18, 2005
Posts: 165
Member
|
 Posted:
Apr 30, 2007 - 04:17 PM |
 
|
| Post subject: Googlebots killing bandwidth with shorturls enabled. |
I've enabled shortulrs now for about a month and a half, and I really like how the website is easier to search and things look better on the pages. However, the amount of bandwidth being used by the site has shot up dramatically, from around 10 gb a month to over 40. I'd like to think that this was all due to increased traffic, but I know that isn't the case. My hosting provider contacted me the other day and singled out googlebots as the culprit. Apparently, they've been sucking up tons of bandwidth, so my host blocked their ips via htaccess. Bandwidth usage has gone down, but I'm worried that I won't be indexed very well with google. I like the look with shorturls, but I'd also like to be indexed on google. Is there a way to enable shorurls without having googlebots go crazy on my site? |
|
|
|
|
MDMods.com
MD user level 5
Joined: Jan 06, 2007
Posts: 55
bannato
|
| Posted:
May 03, 2007 - 04:33 AM |
|
|
If it's the first time you've been indexed by google this may well be normal depending on the size of your site. Googlebots go everywhere, which can be a problem on some sites as they're none too smart and can go round and round in circles.
Sadly without looking at your access logs no one will be able to tell you exactly what went wrong when they attempted to index you. |
_________________
MDMods for all your MDLite module needs |
|
|
|
|
NateM
MD user level 5
Joined: June 18, 2005
Posts: 165
bannato
|
| Posted:
May 03, 2007 - 11:39 AM |
|
|
I've been indexed on google, and indexed fairly well, for quite some time, so it isn't that. I'm not at my home computer right now, so I can't really access my logs, but when I get home I'll try to pick through something. Thanks for the help so far. |
|
|
|
|
|
|
|
| |
Forum FAQ
Search
Usergroups
Preferences
Options forum
Watched Topics
Latest forum posts
Log in to check your private messages
Log in
