News:

Wondering if this will always be free?  See why free is better.

Main Menu

Search Engines

Started by Telekinetic, May 04, 2016, 01:01:46 PM

Previous topic - Next topic

Telekinetic

So being the stupid person I am, for no reason I typed mywebsiteurl into the google.com search bar, and I was the first result.
It said "oh no" under it my URL and under it "A description for this result is not available because of this site's robots.txt – learn more"
Being even a more retard, I went to bing to do the same. It said this "We would like to show you a description here but the site won't allow us." How to fix this?

Illori

what is your robots.txt file? it should be in your main SMF folder.

Kindred

Didn't you just say in another thread that your site does not allow guests?

How would google actually catalog anything if you don't allow guests?
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

Arantor

Especially if you ban them...

Telekinetic

I don't know. I have Guest browsing forum disabled. I have never touched Robots.txt, and Google still goes past the login. I'm lost.

Illori

maybe if you actually provided us with the URL of your forum we can actually answer your questions and not guess as to the answer.

Telekinetic

Quote from: Illori on May 04, 2016, 01:38:20 PM
maybe if you actually provided us with the URL of your forum we can actually answer your questions and not guess as to the answer.
http://dystopia.tk/

qc

#7
I had a quick look at your site's HTML and HTTP headers, which are pretty standard. There is also no robots.txt available. Therefore google or bing should crawl your site - but since they don't, did you recently delete a robots.txt from your webroot?

Edit: Seems you had a robots.txt similar to this one http://www.dystopias.tk/robots.txt before you removed it (your www subdomain seems to redirect to advertising, probably a feature of your freehoster).
Playing quizduell? Having quizduell questions? Our german quizduell forum quizcommunity.de is looking for quiz freaks to come and play quizduell with us :)

Telekinetic

Quote from: qc on May 04, 2016, 07:23:32 PM
I had a quick look at your site's HTML and HTTP headers, which are pretty standard. There is also no robots.txt available. Therefore google or bing should crawl your site - but since they don't, did you recently delete a robots.txt from your webroot?

Edit: Seems you had a robots.txt similar to this one http://www.dystopias.tk/robots.txt before you removed it (your www subdomain seems to redirect to advertising, probably a feature of your freehoster).
never deleted anything. I have no knowledge of how to employ the use of robots.txt, and uh how to allow description for my website when it is searched up. Help me, if you can.

qc

Then I am pretty sure the previous owner of dystopia.tk (at least since 2014) had a robots.txt in his web root blocking all search engines. You will now simply need to wait till Google & Co re-index your site. You might try to submit your website to e.g. Google for re-indexing.
Playing quizduell? Having quizduell questions? Our german quizduell forum quizcommunity.de is looking for quiz freaks to come and play quizduell with us :)

Advertisement: