Uutiset:

Wondering if this will always be free?  See why free is better.

Main Menu
Advertisement:

Crawlers cannot spider my site

Aloittaja pope and young, marraskuu 30, 2008, 09:04:34 AP

« edellinen - seuraava »

pope and young

I made the plunge and went with smf version 204, I just rebuild the forum from the ground up since I had no members anyways.

Anyways, I tried creating a site map with some software I have and it only pulled up one "url" My home page.

I tried an array of online site map creators and the same thing! I even went as far as download different software.

What it comes down to is my site cannot be crawled for a site map, does this mean google is going to have trouble crawling my site also?

I hope I posted this in the right place.

P.S. I know my software works because I used it on smf version 117

shadow82x

Hello,
I have moved this to support as it does not appear to be a bug. Active 2.0 forums have active crawlers. Hopefully the support team can help you. :)
Colin B
Former Spammer, Customize, & Support Team Member

swtdivalove

If you want to create a sitemap for crawlers, I suggest using Google's Sitemap Software.  It works.

Also, you may not have had logging of spiders/crawlers turned on.

I see Google on my site many times a day right now.

There are also other things that are accessing your site beyond spiders.

greyknight17


pope and young

Thanks for the replies, I got godaddys site map creator to work, but the others won't work.

Doesn't that strike anyone as strange how my site map software can no longer crawl my site since the switch to smf 204? Plus the loads of online sight map generators don't work now?

The thing about me is, I  go through my websites with a fine tooth comb so that is how I find little things that may be overlooked.

H

Sorry for the slow response. Do you still require assistance? If so, please post a link to your site. I tried the one in your signature, but that seems to be a godaddy error page :(
-H
Former Support Team Lead
                              I recommend:
Namecheap (domains)
Fastmail (e-mail)
Linode (VPS)
                             

karlbenson

You are better using smf sitemap generater rather than any third party generator (including googles or godaddys)
because they use up too much resources.

They have to literally pull every page to find all the links for your sitemap
(which on several occasions has resulted in the host suspending the site for abuse [equivalent to a ddos])

SMF sitemap generator on the other hand simply queries the database and then builds the sitemap.
No thousands of page requests, no waste, no inefficiency. (and quick).

greyknight17

pope and young, is this issue resolved now?

Advertisement: