Meta Says "noindex" in robot

[club2400]

Meta Says "noindex" in robot, which I assumed was not allowing the bot to scan the index, so I deleted it and left nothing there. Am I wrong? Thanks

[Dannii]

You should leave it in so that it can do it's job. It stops the robots from indexing duplicate content, leaving only a single copy of each page.

[club2400]

Thanks a lot.

[freemn78]

What do you think about

Code Select Expand

<meta name="description" content="', $context['page_title'], '" />', empty($context['robot_no_index']) ? '' : '
<meta name="robots" content="index, follow" />', and a robots.txt with:

Code Select Expand

User-agent: *

Disallow: /*msg

[Harro]

I do believe not all spiders obey the wildcards.
Not sure about it though...

[freemn78]

But the important ones like yahoo & google will accept it ?!?
My Problem: if I leave the robots tag like original (noindex) google indexes all pages from my domain; but not the topics from the boards; if I´ll change it without the robots.txt modification I´ll get duplicate content 

[Harro]

Like eldʌkaː said, you shouldn't change it all.
It's just to prevent duplicate content, but your topics will still be indexed by the search engines.
You just won't recieve a penalty for having duplciated.
It really is best if you just don't touch that.