Robot_no-index code

[Dannii]

I am still having a similar problem, Google will not move past my index page. All other bots look into boards except Google and Google mobile. I was told it was because of Portamax, but I have had this problem ever sens I switched to SMF2.0 RC1 with no mods. I have almost become unknown to Google because of this. What can I do?

In QueryString.php find this line:

Code Select Expand

// There should be a $_REQUEST['start'], some at least.  If you need to default to other than 0, use $_GET['start'].
Add before it:

Code Select Expand

print_r($_GET);
This will print out the GET array. Visit the problem pages on your forum, copy the array from the source of those pages and post them here, and then delete the line you just added.

helpdivya, you may want to do the same, just to check it's working, and that no other mods are causing problems.

Once you make the changes the search engines will have to spider your site again before your pages start getting listed. noindex doesn't mean never index, but it could take the usual couple of weeks for the spiders to come by again.

For the wap, there was the idea of usually having noindex, but allowing indexing if it was a known mobile spider. Do you know if anything came of that?

[karlbenson]

Dannii, in response to your last part.
No. its an impossibility. You can't show normal content to normal spider and the same mobile content to mobile spider.
its still duplicate content.

In 2.0 RC2 when released, we will be using Canonical tag on mobile pages to page the pagerank to the normal version. (which also tells them only to index the normal version).

[Dannii]

Well you could also show noindex on all non-wap/imode pages to known mobile spiders...

Ahh canonical, yes that should work well.

[karlbenson]

That won't make a difference.
When checking for duplicate content, it compares all pages in the index, not just ones discovered by the same type of spider.

[Dannii]

0.9.3 is now released which includes the fix for this.

[karlbenson]

Thanks Dannii.

[helpdivya]

Thanks once again both of you.

So what the final outcome? and this is what I intend to do.




• Change the wireless.template.php file as mention by regularexpressio to make sure that wap, wap2 pages doesn't get index. Please let me know if there is something more to do.
• Find a OOB index.template page and revert the robot_no_index to noindex
• Install the new version of version of pretty to URL that Dannii just released.
• 
  

Please let me know if there is anything else that I need to do.

And Dannii from what I understand is that this new release will fix following issue in RC2.




• All the topics, boards, child boards, profile page will get index.
• It will now prevent search crawler from indexing duplicate pages across the forum
• No more index of printpage

Also could you please tell me do I need to uninstall the current version of pretty URL and Install the new version or I can just install on top of the current version.

Thanks for your help and time. Waiting for your reply.

[Dannii]

It will just restore the normal SMF noindex functionality. I can't remember exactly which pages are noindexed, but the topics and boards and profiles will all work.

Yes, uninstall the old before installing the new.