Guests still browsing forum, but allow is unchecked

[walkerp]

I am attempting to restrict bots and spiders from crawling through our forum.

I unchecked "allow guests to browse through forums".



And yet I still see this:



what gives?

[Arantor]

That means they're *trying* to access it but they're not succeeding. It doesn't verify that they *can*, merely logs what they're trying to do. If it were to verify they could do so, it would make everything a lot slower.

[Illori]

just because you see they are *trying* does not mean they are actually getting access.

test it yourself to verify.

[walkerp]

But how can they even get to those pages in the first place?

[Arantor]

They're NOT getting to them, that's the point. They have links somehow already (or are just brute-forcing every topic), and there's various ways that can happen.

[walkerp]

The same forum on the same url used to allow guests, so I guess they could have already accumulated a list of all the sub-forums and then are going directly to them.    So theoretically, any new topics, they wouldn't be able to find?

Anyhow, thanks for the prompt responses.  I guess I should learn more how these bots and spiders work.

[Arantor]

Yup, nor would they be able to actually *get* to the topics you've now hidden.