News:

Wondering if this will always be free?  See why free is better.

Main Menu

googlebot.txt ?

Started by sombra, November 15, 2008, 03:40:29 AM

Previous topic - Next topic

sombra

Analyzing URL: Open Window to Visit   http://.....net/
Loading...

Note: This site does NOT provide the robots.txt file or the robots.txt file is not constructed properly. It is recommended all sites provide this file.

is this a problem of smf that i need to fix  in smf or is a problem in my host?

can someone give me a hand here hehe

how can i fox this

sombra

im using 2.04 smf if that helps is there any way to make my forum bot friendly

H

Please post a link to your site.
-H
Former Support Team Lead
                              I recommend:
Namecheap (domains)
Fastmail (e-mail)
Linode (VPS)
                             


H

Your site doesn't have a robots.txt. This isn't essential but it can help search engine rankings in some circumstances. Search this board for robots.txt for some example entries :)
-H
Former Support Team Lead
                              I recommend:
Namecheap (domains)
Fastmail (e-mail)
Linode (VPS)
                             

sombra

mmm im a newbie can you tell me if there is some place were i can get one already made?

H

You can't get ready made files because it depends what you're running on your site.

If you want one for SMF, just search this site for robots.txt. You'll find lots of examples. Then you just put them in a text file (using Wordpad/Notepad etc) and then save the file as robots.txt.

Then just upload it to your website using FTP. See: How do I use FTP? / What is FTP?
-H
Former Support Team Lead
                              I recommend:
Namecheap (domains)
Fastmail (e-mail)
Linode (VPS)
                             

sombra

#7
Quote from: H on November 16, 2008, 08:47:24 AM
You can't get ready made files because it depends what you're running on your site.


it depends on my host or my web?

and how come my site dosent have a robot.txt its normal in a smf instalation? or im a rare case?

sombra

#8
i found this one can some one help me out to fix it i dont know to tweek it hehehe

for my site i dont care if google looks for all my site i just want him looking in my forum does this one work for me?

sombra

i found this site http://www.xml-sitemaps.com/ to create sitemaps for google i want to know if it works heres the sitemap that it created for me if someone can look at it and tell me if its fine


H

Quote from: sombra on November 16, 2008, 12:45:33 PM
i found this site http://www.xml-sitemaps.com/ to create sitemaps for google i want to know if it works heres the sitemap that it created for me if someone can look at it and tell me if its fine

I'd recommend the Sitemap Mod instead. It will dynamically update and won't cause bandwidth to get used up when it updates, like an external sitemap creator would.

Quote from: sombra on November 16, 2008, 10:02:16 AM
i found this one can some one help me out to fix it i dont know to tweek it hehehe

for my site i dont care if google looks for all my site i just want him looking in my forum does this one work for me?

That won't help for your site. Please do as I have suggested and run a proper search for robots.txt on this site. Not that a robots.txt is a nice extra, and is not required.
-H
Former Support Team Lead
                              I recommend:
Namecheap (domains)
Fastmail (e-mail)
Linode (VPS)
                             

sombra

im using smf 2.04 that mod is comatible? and im using darkbreak theme

LOADING...

Quote from: H on November 17, 2008, 01:18:20 PM
Quote from: sombra on November 16, 2008, 12:45:33 PM
i found this site http://www.xml-sitemaps.com/ to create sitemaps for google i want to know if it works heres the sitemap that it created for me if someone can look at it and tell me if its fine

I'd recommend the Sitemap Mod instead. It will dynamically update and won't cause bandwidth to get used up when it updates, like an external sitemap creator would.

Quote from: sombra on November 16, 2008, 10:02:16 AM
i found this one can some one help me out to fix it i dont know to tweek it hehehe

for my site i dont care if google looks for all my site i just want him looking in my forum does this one work for me?

That won't help for your site. Please do as I have suggested and run a proper search for robots.txt on this site. Not that a robots.txt is a nice extra, and is not required.

Thank you for the recommendation

sombra


Deprecated

Please read these two sites to gain an understanding of the subject:

http://www.robotstxt.org/

http://en.wikipedia.org/wiki/Robots.txt

sombra

thanks helped alot :D i just made my robot.txt and now i know whats its for

but im having some trouble understanding the sitemaps since i tried to install the mod and i cudent i went again to a site so i can make it for me

i know that its like an index of my web but im asking my self do i need to update it periodically? if i dont what woud happend?

Deprecated

Use "SMF Sitemap" and you won't have to update anything manually. That's what I use.* Unfortunately it is not clear from the Mod Site if SMF Sitemap works on SMF 1.1.7 or SMF 2.0 Beta 4. Perhaps the support topic has details.




*note: I've modified my Sitemap so my experience does not indicate whether it works with Beta 4.

sombra

thanks for the reply im trying waiting on there reply but still i have the big question hehe if i or someone make the sitemap from an external page it will need to update frequently?

H

If you make the sitemap externally, then yes, you'll need to update it every time you get a new post / member.

Obviously it isn't crucial if it gets a little outdated, but it wouldn't give any benefit.
-H
Former Support Team Lead
                              I recommend:
Namecheap (domains)
Fastmail (e-mail)
Linode (VPS)
                             

Advertisement: