News:

Wondering if this will always be free?  See why free is better.

Main Menu

robots.txt and addon domain question

Started by pepf, March 05, 2020, 01:40:32 AM

Previous topic - Next topic

pepf

Domain setup:    smfforum.addondomain.com points to the folder maindomain.com/forum, with SMF placed in the 'forum' folder.
Access should be via the addondomain.com.

Should the robots.txt be
Disallow: /index.php?action=profile*
or
Disallow: /forum/index.php?action=profile*
or both, just in case a spider finds maindomain.com/folder/

Thanks in advance

drewactual

your first one should be sufficient. 

here's to hoping they actually honor it, right?


Kindred

or just turn off profile access for guests
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

pepf

Thank you both. Turned guest access off.
Didn't even know that setting exists. Just noted that two SE spiders were visiting the profiles.

Advertisement: