SMF Support > SMF 2.0.x Support

Robot txt

(1/1)

rambomambo:
Hi to all need good robot text for SMF 2.0.2 got this one from Optimus Brave


--- Code: ---User-agent: Googlebot-Image
Disallow: /

User-agent: YandexImages
Disallow: /

User-agent: msnbot-MM
Disallow: /

User-agent: Googlebot-Mobile
Allow: /*wap
Disallow: /

User-agent: YandexImageResizer
Allow: /*wap
Disallow: /

User-agent: MediaPartners-Google
Allow: /

User-agent: *
Allow: /$
Allow: /*action=.xml
Allow: /*sitemap
Disallow: /*action
Disallow: /*wap
Disallow: /*board=*wap
Disallow: /*topic=*wap
Disallow: /*topic=*.msg
Disallow: /*topic=*.new
Disallow: /*;
Disallow: /*PHPSESSID
Allow: /*board
Allow: /*topic
Disallow: /
--- End code ---

mrintech:
what's wrong with that? ???

MrPhil:
If you do a search on this forum, you will find a number of discussions of what should go in a good /robots.txt file for SMF.

Navigation

[0] Message Index