The Robots META tag is widely used in SMF to prevent indexing a page multiple times.
It is to prevent that a forum topic is getting indexed multiple times but with different query string or the addition of an anchor/fragment.
Typically the URL
http://chaimai.com/index.php?topic=1594.0
will be indexed but the URL
http://chaimai.com/index.php?topic=1594.0;prev_next=prev#new
will probably not be indexed because it´s using the directive <meta name="robots" content="noindex" /> in its HTML.
I'm wondering though, if Google and other search engines distinguish between the two URL's above.
What if Google marks and remebers the path only .... that would block future indexing of the main page http://chaimai.com/index.php and as a result, all the pages in the SMF forum ..., it would be a disaster for someone working with SEO :(
http://chaimai.com/index.php?topic=79.0;prev_next=next#new
\___/ \_________/\________/\________________________/\___/
| | | | |
protocol host path query anchor/fragment
Another way to ask the same question: Is the robot directive referring to the whole URL with path, query and anchors or only to the path?