Simple Machines Community Forum

SMF Support => SMF 2.0.x Support => Topic started by: petewadey on January 09, 2018, 06:06:50 AM

Title: Search Engines
Post by: petewadey on January 09, 2018, 06:06:50 AM
I have Search Engines as a member group and given them certain permissions, somewhere between Guests and Members.
Most bots turn up in this group as if by magic, obviously something SMF do behind the scene. How often is this updated?
I ask because when I check guests ip's, I sometimes find genuine bots, and add them manually to the search engine group.
Mainly because I have given them more permissions than they would otherwise have as a guest.
Am I wasting my time?, Should I just be patient and wait for the data to update?
Thanks
Pete
Title: Re: Search Engines
Post by: Illori on January 09, 2018, 06:21:47 AM
not all bots will crawl your site and define themselves as a bot. if they dont they will not let the forum put them in the group. IMO you will hurt your ranking more if you give spiders more access then a guest gets.
Title: Re: Search Engines
Post by: Kindred on January 09, 2018, 07:08:00 AM
it actualy violates Google's terms to have the bot see more than a standard guest.
Title: Re: Search Engines
Post by: petewadey on January 09, 2018, 08:24:31 AM
Sorry, I was wrong in my original message. I allow guests to see that pages are there for members only, but don't wish bots to see those. That is the only difference.
Can anyone answer my original message please?
Title: Re: Search Engines
Post by: Illori on January 09, 2018, 08:26:25 AM
since spiders dont always identifiy themselves you will not always get what you want from that. as if they dont identify they will see the pages and see when they do they cant and down rate your site.
Title: Re: Search Engines
Post by: petewadey on January 09, 2018, 08:33:04 AM
In that case, I'd better rethink my permissions. Thanks for your help
Title: Re: Search Engines
Post by: Kindred on January 09, 2018, 03:11:48 PM
Additionally, if I can see something, I should be able to get to it.
If I click on something and the system tells me "you have to register to complete this action" - I leave the site and never come back.