Googlebot & Spiders

EG · March 03, 2006, 02:39:06 PM

are you viewing your site with a theme other than the default theme ?

you'll have to adjust your own themes accordingly, as the package only modifies the Default SMF theme

Prasad007 · March 03, 2006, 05:02:39 PM

i know that

im using defult only

EG · March 03, 2006, 06:25:36 PM

fair enough, worth checking

Bigguy · March 03, 2006, 06:26:03 PM

Have you looked to tinyportal.net to see if they have any answers for the sitemaps

As for this mod I think just waiting for them to come and spider you is what you`ll have to do. One other thing have you changed the meta tags to discribe your site a bit more

Quote from: Prasad007 on March 03, 2006, 01:55:45 PM
yes i do have the sitemaps mod installed.but it doesnt function properly due to TP being installed.

Omar Bazavilvazo · March 04, 2006, 12:25:54 AM

try using firefox and user-agent extension, as i explained previously on this topic

also adding your url to google would be a good idea
http://www.google.com/addurl/?continue=/addurl

Prasad007 · March 04, 2006, 05:43:59 AM

i have the meta tags and meta descrption set perfectly. thats not a prob.
and till date i havent found a solution to the TP - sitemaps integration.

DemonicInfluence · March 04, 2006, 12:29:38 PM

Meta tags don't really work with google nowadays...

The sitemaps integration problem is only the html sitemap.. The google sitemap still works..

Prasad007 · March 04, 2006, 08:57:17 PM

so what can be done to ensure spidering of my site ?

DemonicInfluence · March 05, 2006, 05:53:58 AM

ummm... Nothing but waiting... and linking to good sites..

However to test if the this mod works is easy.. Just try to validate with xhtml validator..

Brakdûr · March 05, 2006, 06:45:06 AM

Hello,

maybe I´m to stupid, but I can´t find the page to set up the Mod (like the pics you added in the modification section).

Can anybody tell me where to find the settings? ( I have searched the whole settings on my page^^)

Greetings

DemonicInfluence · March 05, 2006, 03:40:34 PM

What smf version + mod version?

nokonium · March 05, 2006, 03:48:44 PM

Could this be modified to identify, as a seperate category, existing members who haven't logged in or who have recently logged in and other specific guests like ones who camp or banned members. I have no problems about having to create a custom field or array etc.

Brakdûr · March 05, 2006, 06:38:31 PM

Quote from: DemonicInfluence on March 05, 2006, 03:40:34 PM
What smf version + mod version?

If you quoting to me:

SMF 1.1RC2

Mod 1.05

But why do you ask? I don´t think this is a matter of versions.

Greets

DemonicInfluence · March 05, 2006, 08:17:46 PM

Quote from: Brakdûr on March 05, 2006, 06:38:31 PM
Quote from: DemonicInfluence on March 05, 2006, 03:40:34 PM
What smf version + mod version?

If you quoting to me:

SMF 1.1RC2

Mod 1.05

But why do you ask? I don´t think this is a matter of versions.

Greets

BEcause umm.. In smf rc2.. You have to to features + settings and then press the link to googlebots..

djalecc · March 08, 2006, 06:59:59 AM

Hi

I like the Idea of this MOD, have installed it in a test place before going further.

Is there any plans for you to add to the permissions, so that the bots can crawl the entire site using permissions, thus the bots get access to the entire forum, but if a guest tries to access restricted pages then they have to log in or register?

I've seen this feature on other forums, when your searching in google for something, it lists the page with the info on it, yet when you click 'next' or 'reply' then it doesnt go any further and you have to register.

Just an idea, it would be good if google could crawl your entire forum, but guests could not.

What do you think?

DemonicInfluence · March 08, 2006, 11:32:45 AM

Nice. That would be a good idea

.. BUt couldn't then guests use user-agent switcher to pretend to be a bot?

Omar Bazavilvazo · March 08, 2006, 11:58:42 AM

Quote from: DemonicInfluence on March 08, 2006, 11:32:45 AM
Nice. That would be a good idea .. BUt couldn't then guests use user-agent switcher to pretend to be a bot?

Who in his mind will do that? what for? i don't see why a guest will do that.

djalecc · March 08, 2006, 12:01:47 PM

Would this matter?

The purpose would be to capture what is in the post for the search engines, to draw more users to the board...

If a guest has found that page, they are more likley to join up and participate in your forum because they have found what they are looking for?

DemonicInfluence · March 08, 2006, 12:48:59 PM

Quote from: Omar Bazavilvazo on March 08, 2006, 11:58:42 AM
Quote from: DemonicInfluence on March 08, 2006, 11:32:45 AM
Nice. That would be a good idea .. BUt couldn't then guests use user-agent switcher to pretend to be a bot?

Who in his mind will do that? what for? i don't see why a guest will do that.

A guest who wants to see more than they are allowed to?

Quote from: djalecc on March 08, 2006, 12:01:47 PM
Would this matter?

The purpose would be to capture what is in the post for the search engines, to draw more users to the board...

If a guest has found that page, they are more likley to join up and participate in your forum because they have found what they are looking for?

Well, than why bother to limit certain sections for certain membergroups because they can just pretend to be google and see them anyways... If a spider can see as much as say a user who has a lot of power... If you do that, why even bother to hide them from guests?

But I can see a way that would be nice, to have an IP range allowed for spider permissions because all bots operate on a specific IP or IP range

Kindred · March 08, 2006, 01:48:33 PM

In addition, this is a bad idea.

Google insists that anything they crawl is accessible to the public. Allowing google to crawl portions of a site which require registered access makes google not like you (and pisses off the people who are trying to get to the information on your site!)

News:

Googlebot & Spiders