Simple machine & Search engines???

[onedodd]

Can this forum be indexed?

I notice non of these forum categories or threads are indexed by Google. At least that I checked.
The individual forums are indexed by MSN but not the individual threads. I only checked the General forum category and 1 other here but neither were on Google or yahoo but were on MSN. So the Forums can be indexed I guess but the subjects or threads within the main category appear on no se's. And since thats where all the content is so I think indexing of the individual subject threads and replies are most important. Is there a mod for this so the threads can be crawled and indexed by the SE's? Sort of like like phpBB has. 

Any feedback on this?

Thank You.

[Ben_S]

http://www.google.com/search?hl=en&lr=&safe=off&c2coff=1&q=allinurl%3Awww.simplemachines.org%2Fcommunity%2Findex.php%3Ftopic%3D&btnG=Search

Results 1 - 2 of about 30,200 for allinurl:www.simplemachines.org/community/index.php?topic=. (0.62 seconds)

That should answer your question

Bear in mind that this board doesn't even have the SE friendly url's option on.

[Trekkie101]

MSN crawls my site perfectly, 400MB bandwidth this month alone for MSN and it trys to print topics aswell..... and its been getting me on avergae 160 links a month.

[Ben_S]

MSN bot , I hate that thing 3.94 GB so far for me this month. Mr Googlebot is a bit more relaxed 724.82 MB.

[onedodd]

Since you guys are talking about it can some one tell me what tool you use to track the bandwidth of the individual bots that crawl your site?

Thanks

[Ben_S]

http://awstats.sourceforge.net/

Make sure to keep upto date if you intend to use it, they have had a few security issues recently.

[Trekkie101]

Same for me. 

Although I wanna try urchin they look cool.

[Ben_S]

Never liked Urchin, google have just bought urchin though...

[Trekkie101]

Yeah thats how I found out about them.