News:

Bored?  Looking to kill some time?  Want to chat with other SMF users?  Join us in IRC chat or Discord

Main Menu

httpBL

Started by Diego Andrés, February 17, 2010, 03:55:54 PM

Previous topic - Next topic

giveaway365.com

Snoppy Sir,
I have a suggestion.....
Instead of having this warning page, can we use a decent captcha which is available freely and user can put his logo and the captcha only to proceed.

In this warning page ... it seems that our website has a virus and good real user dont return...

So instead of writing so much about the virus and torgan horses , we just welcome them with our logo and below our logo a simple captcha .
Other conditions remain same like if the level is above 50 dont even show captcha etc etc.

lc62003

Quote from: snoopy_virtual on March 06, 2010, 08:32:47 AM


Wouldn't be a bad idea though to add this value to the config page also. I have it in 2 minutes now because all the bots in my forums are hitting like that, but this could be a nice improvement for lc62003



Is this hard coded, or is there something I'm missing.  If this is coded I can go in and change it to see what happens....right now is the 24hrs on for the bot.   :)

Sudhakar Arjunan

Hi Snoopy,

Good work, read the pm already.

And so far, Spammers stopped by MOD httpBL: 1198

Nice work . keep the good work.
Working on New Mods & Themes for SMF... Will update soon... My Blog page
My Smf forum : Discuss ITAcumens :: My SMF Forum

snoopy_virtual

Quote from: butchs on March 06, 2010, 10:29:48 AM
Quote from: snoopy_virtual on March 06, 2010, 08:44:29 AM
[Everytime they hit the honey pot they send a signal to PHPot and everything they do inside it is recorded in their logs, not in yours.

Every page change hit shows up in my cpanel visitor log including honey pot hits.  Yesterday I saw a banned IP get a error message and actually hit my honeypot.  This is why I wonder about the page hits.

Very interesting, so then you can actually check the hits in your honey pot.

I will check my cPanel log looking for that then. I haven't thought about it.

Anyway, think a minute about the way the bots actually crawl the net:

Everytime they enter a new web page they do the things they are programmed for. For example a harvester looks for @, a comment spammer looks for forms with input fields where it can write their spam messages or for links to "register here" to try and register an account in the forum, etc.

But all the bots have one thing in common. They need to store links to other pages to visit them when they finish their work inside the page they are in.

As far as I know, the way they do it is looking for links in all the pages they visit and storing them on a list in their DB. Their "List Of Pages To Visit In The Near Future".

Then the actual flow of events everytime they enter a new page is:

- Look for links inside the page

- Store them in the "LOPTVITNF" I suppose they won't store duplicates, so I think it's no point to put for example 3 links to your HP inside every page.

- Look for damage to do in this page Depending on the kind of bot it is.

- Do as much damage as possible

- Job finished. Visit next page on the list Don't know either if they do that in order or just take one at random. I suppose in order.

So suppose a bad bot have already 10 of your pages on its list (collected from other sites linking to yours it has visited before).

It will try to hit that 10 pages one after the other.

If the mod is working properly the 10 times it will see only the warning page so the only link it will collect from your site is the link to your honey pot. It won't collect any of the pages your page is linking to.

BTW. That's another good thing about the Project Honey Pot. The more people having it on their sites the less possibilities of bots crawling the net. Once a bot arrives to a dead end (a site with no outgoing links) if its "LOPTVITNF" is empty it has nowhere to go.

But, if the bot I was talking before visiting your site, have your honey pot in its list, it doesn't matter if it doesn't visit it today (maybe because it's at the bottom of a very long list) but it will end up visiting it one day.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

snoopy_virtual

Quote from: giveaway365.com on March 06, 2010, 10:41:03 AM
Snoppy Sir,
I have a suggestion.....
Instead of having this warning page, can we use a decent captcha which is available freely and user can put his logo and the captcha only to proceed.

Every bot I know can pass any captcha actually in the net very easily.

There is no one captcha just now that a normal bot cannot pass.

Please read this:

Quote from: Snoopy link=http://www.snoopyvirtualstudio.com/foro/index.php?topic=297.msg1230#msg1230 date=1267473850
It doesn't matter what captcha you put on a site. The robots always pass them without a problem and they are only a pain for the humans.

I have tested a program which can even pass the more complicated captchas I have found (and I have seen a lot of them).

That's why my captcha is so ridiculous, because I want everything as easy as possible for any human visitor. The main point is not the captcha, but all the empty fields I have put inside the form (some of them hidden and other visible). The robot will concentrate on passing the captcha, but I have never seen a spammer not writing on every form field it sees. And that's how I catch them. Human cannot write on hiding fields.

Anyway, of course, this is only a theory. But now I have this new version installed on a few forums I will study the logs and will be able to see if it's right or not.

Quote
In this warning page ... it seems that our website has a virus and good real user dont return...

I spent almost a month to write that warning page. Studying every word, changing them to see if the effect was different, showing it to friends to see their reaction to it, etc.

If you can find a better way of saying all the things that need to be said on this page please let me know and I will change it.

The way the page is just know states very clear "YOUR computer has been doing bad things YOU may have a trojan etc."

It doesn't says at all "OUR site has been doing bad things WE may have a trojan etc."

If anybody reads the opposite it means they cannot read properly.

Quote
... instead of writing so much about the virus and torgan horses, we just welcome them with our logo ...

I thought that was clear in the tutorial:

99% of the visits to your warning page are spam-bots, so the less information on that page they can actually use, the better.

Some of the humans visiting your warning page will be only because an un-lucky dynamic IP, but not too many. Most of them will be because actually they have a trojan and you need to warn them so they can phone a computer tech and clean it.

I love working playing the piano, designing web sites and writing programs, but very few customers actually phone me to do this things. Most of the people phoning me every day is because their computer is not working properly, it's too slow and it's doing funny things and the conversation is always the same:

Me - "It looks like you have a virus"
Customer - "Impossible. I use a brilliant anti-virus"
M - "OK, what anti-virus you use"
C - "This one"
M - "OK, I suppose you update it every day"
C - "Oh. Do you need to update the anti-virus?"
M - [another one] "Mmmm... When was the last time you updated Windows?" [ I already know the answer ]
C - "Oh. Do you need to update that as well? I thought, having automatic updates on ..."
M - "OK, let's see. What anti-trojan you use?"
C - "What was that sorry?"
M - "Doesn't matter. Let me update your system and your anti-virus, install a good updated anti-trojan and clean your system.

As I was saying I love to do more artistic and creative jobs, but as I need to eat, that's what I'm doing all day long.

And every computer I clean I find a minimum of a hundred different trojans, keyloggers, etc etc.

Just check inside google looking for "bot-nets" and you will see how bad is the problem and how many computers are just now zombie-computers working for a bot-net.




Anyway if you are too worried about loosing customers, check your logs and see if you need to change the values in your config page.

For example, if you are getting too many humans with a threat level of 6, 7, or something like it, you can change the settings like this:

From 0 to 10 - let them pass
From 10 to 30 - show the warning page
More than 30 - stop them completely

It's up to you to set the levels to your particular case.

But please remember when I say humans I mean people with 2 good, 0 bad answers. Even 1 bad answer is normal, but too many entries with 2 or 3 bad answers is not normal. Tell me if this ever happens




And of course, if you find a better way to explain the things in the warning page please tell me. I would love this page to be smaller and say the same things with less words, but cannot find a way to do it.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

snoopy_virtual

Quote from: lc62003 on March 06, 2010, 10:45:44 AM
Quote from: snoopy_virtual on March 06, 2010, 08:32:47 AM

Wouldn't be a bad idea though to add this value to the config page also. I have it in 2 minutes now because all the bots in my forums are hitting like that, but this could be a nice improvement for lc62003


Is this hard coded, or is there something I'm missing.  If this is coded I can go in and change it to see what happens....right now is the 24hrs on for the bot.   :)

It's a different thing. You are talking about the cookie for 24h the humans get when they prove they are humans.

BTW, if a friend calls you and say they are answering right the 2 questions and still cannot pass, tell them to turn the cookies on. For next version I'm working on a system that will work even if you haven't got cookies activated, but that's not implemented yet.

The bots haven't got cookies, so if your site is very busy and every bot hits an average of 10 pages in a row, if you need to connect to PHPot database to check their IP 10 times for every one of them, you are loosing too much band-width and server resources, so I had to look for another way to store the values of the response from PHPot database inside your server itself. Not in the visitor computer (as the cookies).

That's what the cache is for.

To have the cache activated you need to go first inside SMF to:

Admin => Server Settings => Caching

In SMF 1.x you will see there that, to be able to activate it, you need to have in your server one of these programs installed and working:

    *  APC
    * eAccelerator
    * Turck MMCache
    * Memcached
    * Zend Platform/Performance Suite (Not Zend Optimizer)

So you will need to ask first your hosting provider which one of those you have (if any) and the instructions on what settings you need to write on the SMF caching page.

For SMF 2 you don't need to have anything special on your server. It works better if you have one of those, but they are using a new system that works even without any of them. Also, by default, SMF 2 has already the "Caching Level " set to "Level 1 Caching (recommended)".

Just go there and check it's on.

For the next version I'm planning to copy their new SMF 2 caching system into my mod for SMF 1.x as well, but that's not done yet. It's only a plan just now.

And yes, the duration of the cache is hard coded and by default SMF have it set at 2 minutes.




Now you know what is the cache, look for it and see if you can turn it on.

In the mean time I'm going to have a break while I think an easy way you could change this value if you need to.

I have been writing here all day long.

The good thing is now in the tutorial, instead of explaining it again I just need to put a lot of links to all this.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

snoopy_virtual

Quote from: A.SK on March 06, 2010, 10:49:26 AM
Hi Snoopy,

Good work, read the pm already.

And so far, Spammers stopped by MOD httpBL: 1198

Nice work . keep the good work.

WOW, 1198 just in one day?

And thanks, I will continue doing my best.  ;)

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

snoopy_virtual

@ lc62003

Let's finish the talk about the cache.

First thing you need to know:

As I said, in SMF the default value for the cache is 2 minutes, but they work in seconds here, so the actual default number is 120. If you don't say how many seconds you want it uses the default 120, so if you write: cache_put_data( whatever ) or you write cache_put_data( whatever, 120 ) it's exactly the same.

As I didn't thought about this when I was doing it, I didn't wrote any number, so that's why the mod is using the default.

Now, if you want to change it to 24h you need to write at the end the comma and the number 60 x 60 x 24, that is, the number 86400

So, first of all activate the cache in your Server Settings.

Then, open the file Sources/httpBL_Subs.php and look for the line:


$response = cache_get_data('httpBL-response-' . $ip2);


Change it to:


$response = cache_get_data('httpBL-response-' . $ip2, 86400);


The line:


cache_put_data('httpBL-response-' . $ip2, 'ok');


Change it to:


cache_put_data('httpBL-response-' . $ip2, 'ok', 86400);


And the line:


cache_put_data('httpBL-response-' . $ip2, $values);


Change it to:


cache_put_data('httpBL-response-' . $ip2, $values, 86400);


Save, upload the file and check if everything is OK.

If you want to experiment with other values, tell me how it goes. Just remember it's in seconds.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

lc62003

Cool!  I may work on that tonight.  It's certainly worth experimentation.  Thanks Snoopy!   8)

giveaway365.com

Spammers stopped by MOD httpBL: 2090

giveaway365.com

sir, is this 2 minute bug anything related to the screenshot I have attached?
Should I make changes or you will update it in new version?

aussieherps

Have upgraded to 2.0rc2 and the httpbl v2.3.1 and there is no /options like originally on the 1.1.11 smf version when I go to the members area.
Any help.
Re-installed scripts and the httpbl mod and still nothing.
Any help.
http://aussieherps.com">snakes lizards spiders turtles and much more

butchs

httpBL_v2_3_4 is the version you should use.
I have been truly inspired by the SUGGESTIONS as I sit on my throne and contemplate the wisdom imposed upon me.

aussieherps

Have just installed v 2.3.4 but still don't see any report buttons or any check IP buttons like in the 1.1.11smf version.
No color coding on the members to say they are OK or anything.
http://aussieherps.com">snakes lizards spiders turtles and much more

snoopy_virtual

Quote from: aussieherps on March 07, 2010, 05:02:32 AM
Have just installed v 2.3.4 but still don't see any report buttons or any check IP buttons like in the 1.1.11smf version.
No color coding on the members to say they are OK or anything.

Let me see:

If I understood correct you haven't installed SMF 2 RC2 like a new forum, but you have upgraded your old SMF 1.1.11 to 2 RC2 and in the old forum you have already installed a version of the mod.

Is that correct? and if so, What version of the mod you had before?




If this is correct, let me do another question:

Did you uninstall all the mods you had in your 1.1.11 forum (as you should) before you upgraded your forum to RC2?




If the answer is yes, another question more:

When you installed httpBL 2.3.4 in your new RC2 forum with your Package manager what exact message you saw on the screen?

Was it saying something like this:

Quote
Installing this package will perform the following actions:
   Type    Action    Description
1.    Execute Modification    ./Sources/Load.php    Test successful
...
etc
6.    Execute Modification    ./Themes/default/languages/Modifications.english-utf8.php    Skipping file
...
etc

With all the lines ending either in "Test successful" or "Skipping file" and none of them ending in "Error whatever"?




Depending on the answers of these 4 questions I will tell you what you need to do now to have the mod working properly.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

snoopy_virtual

Quote from: giveaway365.com on March 06, 2010, 10:25:29 PM
sir, is this 2 minute bug anything related to the screenshot I have attached?
Should I make changes or you will update it in new version?

First thing, it's not a bug at all. You can sort that just activating the cache in your forum.

The problem lc62003 has is completely different and anyway it won't work without cache activated first, so you shouldn't touch your files at all.

Of course, if you cannot activate the cache in your forum you will need to wait for the next version I'm doing.

What I am going to add to the new version is a solution for the people (like myself for example) that cannot activate the cache in SMF 1.1.11 because we don't have in our servers any of the programs needed.

In SMF 2 RC2 you don't need to have any special programs for that, so you can always have cache activated if you want.

Of course you could tell me that a solution will be to upgrade the forums from 1.1.11 to 2 RC2 as it seems aussieherps has done, but WHATEVER YOU DO, DON'T EVEN THINK ABOUT DOING THAT.

SMF 2 is still a beta version (that's what RC means) and it's not stable. It's only for test forums and you should never under no circumstances at all use it for a production site. As SMF says very clearly on their download page.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

aussieherps

Quote from: snoopy_virtual on March 07, 2010, 07:26:57 AM
Quote from: aussieherps on March 07, 2010, 05:02:32 AM
Have just installed v 2.3.4 but still don't see any report buttons or any check IP buttons like in the 1.1.11smf version.
No color coding on the members to say they are OK or anything.

Let me see:

If I understood correct you haven't installed SMF 2 RC2 like a new forum, but you have upgraded your old SMF 1.1.11 to 2 RC2 and in the old forum you have already installed a version of the mod.

Is that correct? and if so, What version of the mod you had before?




If this is correct, let me do another question:

Did you uninstall all the mods you had in your 1.1.11 forum (as you should) before you upgraded your forum to RC2?




If the answer is yes, another question more:

When you installed httpBL 2.3.4 in your new RC2 forum with your Package manager what exact message you saw on the screen?

Was it saying something like this:

Quote
Installing this package will perform the following actions:
   Type    Action    Description
1.    Execute Modification    ./Sources/Load.php    Test successful
...
etc
6.    Execute Modification    ./Themes/default/languages/Modifications.english-utf8.php    Skipping file
...
etc

With all the lines ending either in "Test successful" or "Skipping file" and none of them ending in "Error whatever"?




Depending on the answers of these 4 questions I will tell you what you need to do now to have the mod working properly.
Yes to all of the questions.
2.3.4 was installed after your last post to my question.
2.3.1 was uninstalled first and then I re-installed the honeypot mod on the server aswell.
http://aussieherps.com">snakes lizards spiders turtles and much more

snoopy_virtual

@aussieherps

First of all there is something I didn't understand on your first question. When you said:

Quote
... don't see any report buttons or any check IP buttons ...

What buttons are you talking about? Are you sure you are talking about mod httpBL and you are not confusing it with my other mod Stop Spammer?

Mod Stop Spammer has buttons to check and report members, but mod httpBL don't, as it does it automatically.

Quote from: aussieherps on March 07, 2010, 08:29:54 PM
Yes to all of the questions.
2.3.4 was installed after your last post to my question.
2.3.1 was uninstalled first and then I re-installed the honeypot mod on the server aswell.

It seems when you upgraded from 1.1.11 to RC2 something went wrong and you still have part of your forum using 1.1.11 files or parts of your database have still the 1.1.11 form, completely different than the 2.0 form.

As I thought that was going to be your answer and I already suspected that was the problem, I have been checking, since you asked it yesterday, the SMF downloads section, looking for a package to upgrade a forum from 1.1.11 to 2.0 and there is not one done yet.

Can you tell me which one have you used?

The differences between 1.1.11 and 2.0 inside the database are too big to try and do it manually. So I suppose as soon as they finish the definitive stable version for 2.0 they will do an upgrade package so all of us will be able to move our forums to the new version, but as 2.0 is still in beta versions only suitable for test forums and nobody knows yet how it will be the final stable version, they haven't done yet this upgrade package. Beta versions change too much between one and the next and they change too often so it's no point to do an upgrade package from 1.x to 2.0 until 2.0 becomes stable.




Solutions:

The solution I would recommend is to take the security copy of all the files and the database you should have from your old forum and leave everything the way it was before.

If you didn't make a security copy before upgrading or you don't want to use that solution there is only another possible solution I can see:

Imaging you have your forum now installed for example in http://www.whatever.com/oldforum

Create a new folder in your server with a different name. For example "newforum".

Install in this new folder SMF 2 RC2 from scratch, with the original full new installation package from SMF, so you will have now another forum in http://www.whatever.com/newforum

To be sure this new forum doesn't touch any of the database tables in the old forum and the new installation is fully perfect, it would be better if you use a different database to install it, but you can also use if you want the same database just changing the prefix used for all the tables.

I mean: During installation, when you are asked for the database details you need to write a prefix for all the tables and, if you don't change it, the installer will use the default, that is: "smf_" so when you are installing this new forum be sure to write here something different to the one you used for the old forum. Something like "newsmf_" for example.

Another thing you need to be sure is you are using for this new forum a template compatible with SMF 2 as all the templates for SMF 1.x are not compatible at all with SMF 2

Once this new forum is installed, add here all your mods from scratch as well, and when you check everything is perfect here, ask all your members to register again in this new forum and leave the old forum just on "read only" mode. I mean, close all the posts so nobody can write there anything else.

This way everybody will continue talking in the new forum but you will still have your old forum there so everybody will be able to check old posts and see where the arguments where coming from.

A good idea will be to copy as well the important posts from the old forum to the new one, but that will be too much work for you, so I will suggest you can ask your members to copy themselves their own posts.

This way, as soon as all the important things from the old forum are already in the new one you can delete completely the old forum and forget about it.




Good luck and tell me if any of this is any use to you or if you still has a problem with it.

And, if everything goes OK, tell me as well, as I'm curious now and would like to know how all this ends up.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

Wizzlefits

#158
For those that would like a bit of a visual aid, or map of how Mod httpBL and Mod Stop Spammer work, then see the attached file.

Before anyone asks.
No I didn't include the path from the bot trap links (those hidden links pointing to your Honey Pot trap page).
Simply because once installed, well...you can forget about that part of the mod. It doesn't need watching, tweaking or anything else.
Okay...so I forgot that part till just now.  :-[

But really...the bot links, just direct nasty bots to your Honey Pot page which has only one way out, and rest assured...it's not back to your site.  ;)

K! Intermission is over! Back to work!  ::)

aussieherps

Thanks Snoopy it wouldn't work as the members have already been asked to rejoin when moving from phpbb to smf forum.
Lost a few good members that way.
If the site was older and a bit bigger it would work but its still in its growth stage.
Didn't realise there was so much difference in the tables and databases. I'm still a noob at this coding stuff.
It could have been the stop spammer mod I was thinking of.
Upgrade was a complete install of smf2.0rc2 and uploaded database so that members and posts were not lost so that is the problem and I will just have to wait until the stable version is released.

Thanks again for your help.
http://aussieherps.com">snakes lizards spiders turtles and much more

Advertisement: