More Spiders

[karlbenson]

1.2 - 20th June 2008
o Added a handful of new spiders which have been detected in the past month

[kai920]

In this who.template i don't have a "$known_spiders = array ("
I have a " $known_agents = array ("

Is it the same? Can i had the spiders' list you provided to this who.template?

I am also using the custom who.template.php for 1.1.5 and see the same $known_agents = array (...

Code (who.template.php) Select Expand


        array (
            'agent' => 'sogou spider',
            'spidername' => 'Sogou spider',
            'spider' => true,
         ),


Code (your spiders.txt) Select Expand


array (
            'agent' => 'Sogou',
            'spidername' => 'Sogou',
        ),


I'm guessing if I added 'spider' => true to your spiders.txt, I can then replace who.template.php's array with yours. Would that be correct?

[karlbenson]

looks like it yes.

[LanceDean]

1.2 - 20th June 2008
o Added a handful of new spiders which have been detected in the past month

How about a new .txt file with the updated list for 1.1.x users to update our sub.php file?    Pretty please?

edit:
Or is this list up-to-date?
http://www.simplemachines.org/community/index.php?topic=233636.msg1507179#msg1507179

[ThePro]

its a very very good add on for my forum!

[karlbenson]

@LanceDean
Done! - Added/Updated at that link.

[LanceDean]

Thank you, karlbenson!

[karlbenson]

I forgot when i updated it to update that list aswell.

[SgtMic]

New search engine out with new spiders. Cuil


Here is their info on their Spider

[olabaz]

How do I use the .txt file you attached?

[karlbenson]

@SgtMic
Thanks however, Cuil is already detected.
(their spider is called Twiceler).

In fact you'll notice Sgtmic that Twiceler is banned from alot of the web for robots abuses. (including mine).

@olabaz
The .txt file is basically the same spiders in my mod, but for the Googlebot and Spiders mod (which is for SMF 1.1.x).
You'll need to edit the spiders in Sources/Subs.php, and replace with the list I posted.
You need to edit as per the instruction in the .txt.

[olabaz]

But I can't find the line that it says to find in the txt file

[SgtMic]

@SgtMic
In fact you'll notice Sgtmic that Twiceler is banned from alot of the web for robots abuses. (including mine).

Interesting.

[olabaz]

But I can't find the line that it says to find in the txt file

[karlbenson]

@olabaz
Are you using SMF1.x?
Is the GoogleBot and Spiders mod installed?

The answer to both of the above should be YES.

Then edit
Sources/Subs.php

Find the array of spiders which starts

Code Select Expand

$known_spiders = array (

Then replace them.

[Sudhakar Arjunan]

I got it perfectly working in my 1.1.5 Forum.

Your Notes are extremely helpful Karlbenson.

I have One request.
Could i set something like only admin is allowed to see the spiders strength in board index and other pages.
Other users could see the normal fn. like no of guest and others only.

If this is been already discussed, please take me there.

Thanks for this wonderful modifications Karlbenson.

[Sudhakar Arjunan]

Hi , i would like to update the code work in my Dilber MC theme.

Dilber MC Theme as its own board index template page.

Could you help me to make the code show this much of bots in my Dilber MC forum index page.

[karlbenson]

This mod works for SMF 2.x only.

This mod works on ALL themes, since it only contains database entries.
No edits to files are made by this mod. All it does is add lots of rows to the search engines table in your database.

You need to enable the 'search engines' feature of SMF inside your admin panel.

If you using smf 1.1.x, then you'll need to use Googlebot and Spiders mod.

[Sudhakar Arjunan]

In Default Theme,

120 Spiders, 370 Guests, 16 Users, 0 Users in Chat (9 Buddies)

In Dilbermc Theme,

365 Guests, 17 Users, 0 Users in Chat (9 Buddies)

Does not shows the spiders count and spiders list in dilber mc theme.

[karlbenson]

You'll need to raise the issue separately on smf.
I cannot help you with the edits, since this mod doesn't make any edits, but add to existing smf code.