HEY GUYS ,
officially my first post after installing smf 1.1.13...i have been doinn well so far however i have a probblem with this robots.txt file,i have no idea a specific one to use for my smf 1.1.13 and where to upload it (root directory??..www)...i also need help in explaining what tthe user agent means ,because in some i see google attached to it and in others msn...while researching i found this...my smf is still fresh but i have sitemap already...
User-agent: *
Disallow: /*sort=*
Disallow: /*msg*
Disallow: /index.php?PHPSESSID=*
Disallow: /index.php?action=activate
Disallow: /index.php?action=admin
Disallow: /index.php?action=calendar
Disallow: /index.php?action=emailuser
Disallow: /index.php?action=findmember
Disallow: /index.php?action=help
Disallow: /index.php?action=helpadmin
Disallow: /index.php?action=login
Disallow: /index.php?action=logout
Disallow: /index.php?action=mlist
Disallow: /index.php?action=modifykarma
Disallow: /index.php?action=pm
Disallow: /index.php?action=post
Disallow: /index.php?action=printpage
Disallow: /index.php?action=profile
Disallow: /index.php?action=recent
Disallow: /index.php?action=register
Disallow: /index.php?action=reminder
Disallow: /index.php?action=search
Disallow: /index.php?action=theme
Disallow: /index.php?action=unread
Disallow: /index.php?action=unreadreplies
Disallow: /index.php?action=verificationcode
Disallow: /index.php?action=who
Disallow: /index.php?action=stats
Disallow: /index.php?theme
Disallow: /attachments/
Disallow: /avatars/
Disallow: /Packages/
Disallow: /Smileys/
Disallow: /Sources/
Disallow: /Themes/
Disallow: /index.php?*;wap
Disallow: /index.php?*;wap2
Disallow: /index.php?*;imode
Disallow: /index.php?*rss*
so i need help with the one i should use and where to upload it. :laugh:
Put the robots.txt in the root of your public web folder (public_html or www). User-agent: * should tell them all. If the list of URL strings is what you want to tell robots to not follow, you should be good to go. And welcome to SMF.
:)
thanks a lot buddy....
does it mean this one is recommended and i should proceed and upload it????
The exact contents of your robots.txt depends on what you wish to do with it, but if you search the forums here - you'll find plenty of examples on this file.