Customizing SMF > SMF Coding Discussion

Question of robots.txt??

(1/1)

johnpaul2k2:
 i just installed Optimus Brave and i don't want to make mistakes

1. looking at the first image, do i need to include "meta-tag" (whether html or bbc) or should i leave it like that?



2. Looking at the second image, i dont understand this "robots.txt" very well. i only included the code that someone here shown  on how to block "link Profile" . i also saw many other codes but in order not to make mistakes.


Should i copy all the codes to "robots.txt" box?
are there some i should select from?
or any smf tutorials concerning robots.txt??
here are the following robots.txt i saw in the "Generator of rules"

User-agent: Googlebot-Image
Disallow: /

User-agent: YandexImages
Disallow: /

User-agent: msnbot-MM
Disallow: /

User-agent: Googlebot-Mobile
Allow: /*wap
Disallow: /

User-agent: YandexImageResizer
Allow: /*wap
Disallow: /

User-agent: MediaPartners-Google
Allow: /

User-agent: *
Allow: /$
Allow: /*action=.xml
Disallow: /*action
Disallow: /*wap
Disallow: /*board=*wap
Disallow: /*topic=*wap
Disallow: /*topic=*.msg
Disallow: /*topic=*.new
Disallow: /*;
Disallow: /*PHPSESSID
Allow: /*board
Allow: /*topic
Disallow: /
Crawl-delay: 5
Clean-param: PHPSESSID /index.php
Host: topnichez.com

K@:
As this is a duplicate of this:

http://www.simplemachines.org/community/index.php?topic=422210.msg3399449#msg3399449

Topic locked.

Navigation

[0] Message Index