News:

Want to get involved in developing SMF? Why not lend a hand on our GitHub!

Main Menu

i need help with robots.txt file

Started by Staind, May 17, 2011, 11:35:40 AM

Previous topic - Next topic

Staind

HEY GUYS ,
officially my first post after installing smf 1.1.13...i have been doinn well so far however i have  a probblem with this robots.txt file,i have no idea a specific one to use for my smf 1.1.13 and where to upload it (root directory??..www)...i also need help in explaining what tthe  user agent means ,because in some i see google attached to it and in others msn...while researching i found this...my smf is still fresh but i have sitemap already...
User-agent: *
Disallow: /*sort=*
Disallow: /*msg*
Disallow: /index.php?PHPSESSID=*
Disallow: /index.php?action=activate
Disallow: /index.php?action=admin
Disallow: /index.php?action=calendar
Disallow: /index.php?action=emailuser
Disallow: /index.php?action=findmember
Disallow: /index.php?action=help
Disallow: /index.php?action=helpadmin
Disallow: /index.php?action=login
Disallow: /index.php?action=logout
Disallow: /index.php?action=mlist
Disallow: /index.php?action=modifykarma
Disallow: /index.php?action=pm
Disallow: /index.php?action=post
Disallow: /index.php?action=printpage
Disallow: /index.php?action=profile
Disallow: /index.php?action=recent
Disallow: /index.php?action=register
Disallow: /index.php?action=reminder
Disallow: /index.php?action=search
Disallow: /index.php?action=theme
Disallow: /index.php?action=unread
Disallow: /index.php?action=unreadreplies
Disallow: /index.php?action=verificationcode
Disallow: /index.php?action=who
Disallow: /index.php?action=stats
Disallow: /index.php?theme
Disallow: /attachments/
Disallow: /avatars/
Disallow: /Packages/
Disallow: /Smileys/
Disallow: /Sources/
Disallow: /Themes/
Disallow: /index.php?*;wap
Disallow: /index.php?*;wap2
Disallow: /index.php?*;imode
Disallow: /index.php?*rss*

so i need help with the one i should use and where to upload it. :laugh:
24 hrs a day...24 bottles in a crate what a coincidence???

mashby

Put the robots.txt in the root of your public web folder (public_html or www). User-agent: * should tell them all. If the list of URL strings is what you want to tell robots to not follow, you should be good to go. And welcome to SMF.
:)
Always be a little kinder than necessary.
- James M. Barrie

Staind

thanks a lot buddy....
does it mean this one is recommended and i should proceed and upload it????
24 hrs a day...24 bottles in a crate what a coincidence???

Aleksi "Lex" Kilpinen

The exact contents of your robots.txt depends on what you wish to do with it, but if you search the forums here - you'll find plenty of examples on this file.
Slava
Ukraini!
"Before you allow people access to your forum, especially in an administrative position, you must be aware that that person can seriously damage your forum. Therefore, you should only allow people that you trust, implicitly, to have such access." -Douglas

How you can help SMF

Advertisement: