News:

Join the Facebook Fan Page.

Main Menu

httpBL

Started by Diego Andrés, February 17, 2010, 03:55:54 PM

Previous topic - Next topic

snoopy_virtual

Quote from: Sea Mac on October 11, 2010, 09:36:38 PM
snoopy, is it your MOD, or the OS_Browser_Detection MOD, (Or both), that are incompatible with that old Who.template.php MOD?

I also have the page Who.template.php changed to see Users/Guests/Spiders separately in some of my forums so I know what are you talking about.

I cannot understand exactly where is the problem, because both mods OS_Browser_Detection and httpBL are compatible with the page Who.template.php I am using.

Maybe the problem is you have the file Who.template.php inside the folder of the particular Theme you are using. As you know, every time you install a mod all the modifications are done only to the files inside the default Theme folder. Every file inside your particular Theme folder need to be modified by hand.

If that's the case you need to modify manually the file Who.template.php inside the folder of your particular Theme.

Apparently you are using the old one did by Owdy ages ago. As far as I know, modd httpBL is also compatible with that old version, but there is an updated one you can find here:

http://www.simplemachines.org/community/index.php?topic=401980.0

If you are using this last one, I have attached here exactly the same file but with the modifications needed to see OS & Browser if you are using my mod. (Plus 3 more spiders added by me).

Anyway, if you are using any other Who.template.php the 2 modifications my mod try to do are these ones:

Almost at the top of the page, just after the file have opened the function template_main() and declared a few globals, you need to add:



// Include OS & Browser Detection if it's installed and enabled
global $sourcedir, $modSettings;
$os_browser_exists = FALSE;
if (file_exists($sourcedir.'/os_browser_detection.php') && $modSettings['httpBL_view_os_whosonline'])
{
require_once($sourcedir . '/os_browser_detection.php');
$os_browser_exists = TRUE;
}



And then later down the file you will find a place were the first <td> (where you print the name of the user) closes and the second <td> (where you print the time) start:


echo '
</td>
<td nowrap="nowrap">', $member['time'], '</td>


Just before this echo statement you need to add:



// OS & Browser Detection
// Parse the user agent only if OS & Browser Detection is installed and enabled
if ($os_browser_exists)
$os_browser_detected = parse_user_agent($member['query']['USER_AGENT']);
// Display only if allowed
if (allowedTo('view_os_browser') && $os_browser_exists){
if ($os_browser_detected['system'] || $os_browser_detected['browser']){ //Do not display if both are unknown
echo '<br />';
if ($os_browser_detected['system']) {
echo '
<b>', $txt['OS_Browser_OS'], ':</b>
<br /><img src="', $settings['default_images_url'], '/os_browser_detection/icon_', $os_browser_detected['system_icon'], '.png" align="top" alt="', $os_browser_detected['system'], '" /> ', $os_browser_detected['system'], '<br />';
} else {
echo '
<b>', $txt['OS_Browser_OS'], ':</b>
<br /><img src="', $settings['default_images_url'], '/os_browser_detection/icon_unknown.png" align="top" alt="', $txt['OS_Browser_Unknown'], '" /> ', $txt['OS_Browser_Unknown'], '<br />';
}

if ($os_browser_detected['browser']) {
echo '
<b>', $txt['OS_Browser_Browser'], ':</b>
<br /><img src="', $settings['default_images_url'], '/os_browser_detection/icon_', $os_browser_detected['browser_icon'], '.png" align="top" alt="', $os_browser_detected['browser'], '" /> ', $os_browser_detected['browser'];
} else {
echo '
<b>', $txt['OS_Browser_Browser'], ':</b>
<br /><img src="', $settings['default_images_url'], '/os_browser_detection/icon_unknown.png" align="top" alt="', $txt['OS_Browser_Unknown'], '" /> ', $txt['OS_Browser_Unknown'];
}

}
}
                   


As I said, if you are using the latest Who.template.php done by MoreBloodWine in the link I gave you, you only need to replace it with the file I have attached here.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

snoopy_virtual

Quote from: Sea Mac on October 12, 2010, 04:10:27 AM
Quote from: Sea Mac on October 11, 2010, 04:30:25 PM
I think I have another Problem. At least two of my forums aren't using the default theme and, in those forums, no Spammers have appeared in the Log since I installed it. The "Spammers stopped by MOD httpBL" number has not increased either, since then.  But the Old Version was working fine in these 2 forums ....
etc...

Wait ....I think it is working again.
etc...


I have just now 7 forums. 6 of them with SMF 1.1.11 and 1 with 2.0 RC3

All the 7 forums use a custom Theme. None of them use the default and the mod is working fine in them, so that's not the problem.

There is a big difference between the old version 2.3.7 and the new version 2.4

In the old version, every time you had a visitor, the mod checked their IP in the HoneyPot's database and (if it was a bad one) added it to the logs.

The problem is sometimes robots try to open a lot of your pages in a very quick row. That's very difficult for a human, but very easy for a robot. So sometimes you had in your logs a lot of entries, all of them from the same IP, with only a few seconds (or fractions of a second) between them.

That was not happening for the people with cache activated, because every time the mod detected a bad IP, apart from writing it in the logs, it also wrote the data in the cache. So if the same IP tried to open again another of your pages a few seconds later, the mod, instead of checking the data in the DB, checked it in the cache.

The problem is, every time the mod needs to check HoneyPot's DB, it needs to open a connexion from your server to HoneyPot's server, and if you need to do that too often it will slow down your forum too much.

In the new version 2.4 I have added a system to emulate the cache for everybody (using the session as well) so now it doesn't matter if you have cache activated or not.

If a robot try to open 100 of your pages in a quick row, the mod will only check the DB and write the data in the logs the first time.

The next 99 times the robot will be redirected to the warning page without needing to check the DB (because the data is already inside the cache and/or session). So now it works a lot faster and it's not slowing down the forums with a lot of visitors per second.

I have added in the mod's config page also a place so you can control how many minutes you want to keep the data in the cache and/or session. The default is 5 minutes. I'm not sure yet if that's the best number, but I think is a good one.

If you have erased all the entries you had in the logs before, we cannot check if that's what it's happening, but maybe you were having 100 entries a day but it was really only 1 spammer visiting 100 of your pages in a row.

Anyway you may have seen I have added also in this new version 2.4 a system to check if the mod is working properly. It is at the top of the mod's config page. If it is green every thing is OK. If it is red it will tell you what is the problem.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

Sea Mac

Thanks snoopy_virtual!

All 4 of my forums are working perfectly now with these 2 custom versions! I have one forum with Only 2 MODS (OS & Browser Detect and httpBL), and one that is all tricked out with lots of MODS, and those files on this and the previous page work perfectly in all of them!

I LOVE the look of my Who's online page! It's eye candy now - That's really Cool!
(I'm sure that hurts a few people's eyes  8) )

Many, many thanks!    

jrcarr

#363
I have tried to install this mod on a 1.1.9 version of SMF, but seem to have a problem.  Though it seem to install ok, when it transfers you to the configuration page at the end of the install, all of the text seems to disappear.  The form fields are there, some being populated with information, but none of the labels telling what the field represents are visable.  Also, even though there is a space under Members for the link to the configuration page, there is nothing there text wise either.  And last but not least, the option bar above the form has the tabs visible, but no text or link in them either.  It's like the language file is missing or something that fills these fields.  I haven't a clue where to look or what to do.  I tried uninstalling and re-installing the mode, both appearing to work fine, but still no text.  I've attached an image showing what is there or should I say not there. ;D

Thanks for any advice

Jack

Update: I've uninstalled the mod again, upgraded to 1.1.11 and re-installed the mod, but have the same problem.
Jack Carr

Sea Mac

#364
I had a crazy problem one time with missing text. I tried everything ....

I backed up my database first. Then I clicked the button in the Admin panel ---> Forum Maintenance ---->
Convert Forum and Database to UTF-8 and THAT Finally fixed it.

One Question: Is your forum/database already converted to UTF-8 or not?

jrcarr

I'll have to look, I don't have a clue if I converted it or not.  I'll get back as soon as I check.

Jack
Jack Carr

jrcarr

Ok, I backed up my database, did the conversion all looks fine on the website.  So I uninstalled the mod again, and I reinstalled the mod again and still have the same problem, but thanks for the suggestion.

Any other thoughts?

Jack
Jack Carr

Sea Mac

How does the Default Theme look? Is It OK?

I suspect the problem is in your theme and not this MOD ...  :-X

I also suggest you download the SMF 1.1.11 Update package (Backing everything up, again, of course) and upload any changed files. Then run the update.php script.

I had crazy problems with a forum I installed 3 years ago. I faithfully installed each security/version update that appeared in the Admin Panel over the years. The crazy problems continued until one day an error popped up stating that I was using a 1.1.3 database (It was version 1.1.7 by then - years later) and said I needed to update the Database to the latest version!

Well, installing a version update from the admin center link DOES NOT Update the database! I still had the same version database that the forum install script had put in back in 2006! Yipe! Anyone that just updated one or more versions from the Admin Center link/Package Manager (the "2 Click" forum upgrade) SHOULD download the UPDATE 1.1.11 Package and FTP upload all the files in it to your forum. (Close the forum for maintenance first.)

Run the Update.php script to Update your forum database to database version 1.1.11 (Update your forum too.). Keeping up with version updates is Critical! But you need to update your database, too, and only the update.php script does that!

(I need to do that myself, I just realized ... Nuts! FOUR TIMES ....  :o )

jrcarr

OK, when I switched to the default theme, all was well and I was able to complete the configuration and believe it is running now.  But, when I switch back to my custom theme, I have the same problem in the admin section.  I have done all upgrades, including running the upgrade.php, updating all the files from the upgrade package, yada, yada, yada.  The only thing I haven't done yet is to update the index.template.php file with the newest version adding my custom changes to it.  Is there any other template file that might need looking at?

Jack
Jack Carr

jrcarr

Ok, I got it working correctly.  It seems that I must have missed something in the tutorial about the language/modification.template.php needs to be moved to the language folder in the custom theme.

Thanks for all suggestions and help.

Jack
Jack Carr

snoopy_virtual

Hi Jack,

Sorry I didn't see you problem earlier. By the time I have seen it it seems it's already sorted.

This mod should work properly with SMF 1.1.9 but I have not tried it, so I'm not fully sure.

Anyway of course (as Sea Mac suggested) the best thing was to update SMF to the latest version. From 1.1.9 to 1.1.11 had been some important security updates and it was a risk to continue using the old version.

Also the problem is the tutorial is not finished yet (my fault) so the instructions there are not really clear. I hope I will have time to finish it some day. Luckily you managed to do it.

Anyway, it shouldn't be any language files on your custom Theme. All the language files should be in the default folder, that's why I never mentioned that in the tutorial. I need to remember to add that as well to the tutorial just in case somebody has the same problem.

If you get any more troubles let me know.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

EL34

Hey Snoop,
Got a suggestion.

Please add the page index at the bottom of the log pages also.

I find myself looking at all the log entries and then having to go back up to the top of the page to click on the next page link.

Thanks
Forum History -> EZBoard -> YABB -> SMF 1.1.19 -> SMF 2.0.19

snoopy_virtual

I see what you mean. Yeah, good idea.

I will add it in the next version.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

EL34

Forum History -> EZBoard -> YABB -> SMF 1.1.19 -> SMF 2.0.19

Lou69

First, a long overdue

THANK YOU!!

for this wonderful mod. To date, 239 baddies have been trapped. Fantastic!!

My current version is 2.3.4 and I will upgrade to 2.4 soon. Will upgrading delete all honeypot date and trap counts? Anything special I need to do before the upgrade?

thanks

Lou

:)

snoopy_virtual

Quote from: Lou69 on October 28, 2010, 12:02:26 AM
First, a long overdue

THANK YOU!!

for this wonderful mod. To date, 239 baddies have been trapped. Fantastic!!

You are welcome  ;)

Quote from: Lou69 on October 28, 2010, 12:02:26 AM
My current version is 2.3.4 and I will upgrade to 2.4 soon. Will upgrading delete all honeypot date and trap counts?

Just un-install old version and install new one.

If you are using SMF 1.x it won't touch at all the info already in your DB.

If you are using SMF 2.0 there is a new option (when you are un-installing a mod) asking you if you want to erase as well from your DB all the data associated with this mod. Just leave that option un-marked and all the data you already have will be untouched as well.

Quote from: Lou69 on October 28, 2010, 12:02:26 AM
Anything special I need to do before the upgrade?

The only thing really is to check the files inside your custom Theme (assuming you are not using the default one).

This mod (as all SMF mods) only change the files inside the default Theme so if you have any of the needed files in your custom Theme you need to do the required changes manually

In version 2.3.4 the mod was doing an important change inside the file index.template.php to display hiding links to your honey pot inside all your forum pages.

It doesn't matter what Theme you are using, you will always have an index.template.php inside it, so I suppose if you are not using the default (following the instructions on the tutorial) you have already made this change there.

The new version 2.4 do exactly the same change inside index.template.php so if you have already made that change leave it the way it is. Un-installing the old version will only erase that change from the default Theme, no from your custom Theme, so installing version 2.4 you won't need to do the same change again.

But version 2.4 also do a change inside the file Who.template.php (that's new from the old version). So you just need to check if you have this file inside your custom Theme.

It is not normal to have the file Who.template.php inside a custom Theme. Usually that file is only inside the default Theme, but if you have it, let me know and I will give you instructions to edit it manually.

============

Of course if in the middle of the un-installing / installing process you get any kind of error message, don't continue with the process. Copy the error message and let me know.

============

After upgrading check also your forum error log.

Not the mod httpBL log, but the forum one in Admin => Maintenance => Forum Error Log

If you are not using the default Theme you will see there 2 or 3 errors telling you "Required file not found" or something like that, because for a few seconds (in the middle of the process) your forum would have tried to display the hiding link to your honey pot and the required data was not there. (The few seconds just after you un-installed the old version and before you finished installing the new one).

Just erase these few errors and forget about them.

============

That's all really. Don't forget to check the new things in the mod config page. There are a few of them.

Don't forget also to add the new permission to the groups of your members you really trust and you don't want to be checked never by the mod.

To do that go to Admin => Members => Permissions => Permissions by Membergroup choose the group of members you trust and click on Modify. You will see there the new permission.

This is a kind of "white-list" system I have added to the mod (in case you missed the explanations I have written about it where-ever it was I wrote it).

By default this new permission is ON only for the group Administrator and is OFF for the rest of the groups.

============

Check also all the new info there is now in the mod logs.

If you have installed as well the mod OS & Browser Detection (or if you install my small addon to see OS & Browser in httpBL) I think you will like it.

BTW, before you ask, you can install this mod (or my addon) whenever you want. Either before of after upgrading. It's exactly the same.

Also, if you don't want to install this it doesn't really matter. Mod httpBL will work exactly the same. The only difference is without it you will see less info in the logs.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

IdanC

does the Humans Log shows all the users that were asked the varification question, or also those who passed without questioning?

IdanC

one more question if i may (i'm being bombarded with spam the last week, it's a nightmare):

i just got a "girl viagra sex" topic from 95.67.80.146. this ip has a 41 threat level, and the httpbl log shows it was blocked just a few hours ago. how come it got through now?

f.y.i., my settings are -
Number of days to consider good an IP - 10
Threat level considered bad - 23
Threat level considered VERY bad - 40
Minutes to keep the data in the cache - 120   
Hours until an IP needs to prove again it's human - 200

and one last question - is there a way to clear all visitors from a certain range or geographic location?

snoopy_virtual

#378
Quote from: IdanC on October 30, 2010, 09:50:16 AM
does the Humans Log shows all the users that were asked the varification question, or also those who passed without questioning?

Anybody allowed to pass without questioning is not in any log at all.

Inside the humans log is all the people who where questioned and answered properly, proving they were humans.


EDIT: That's not true. Please read answer #382

Quote from: IdanC on October 30, 2010, 10:06:19 AM
i just got a "girl viagra sex" topic from 95.67.80.146. this ip has a 41 threat level, and the httpbl log shows it was blocked just a few hours ago. how come it got through now?

I think it is because all your settings are wrong. Let's see them more detailed:

Quote from: IdanC on October 30, 2010, 10:06:19 AM
Number of days to consider good an IP - 10

That's definitely very wrong.

You are saying you consider that anybody who has not done anything wrong for the last 10 days is now for sure a good guy, so you let them in un-checked by the mod.

Well, suppose this viagra spammer, the first time it arrived to your forum, the last time it has done something wrong was 9 days and 20 hours before. That, for the mod, is the same as 9 days.

The mod checked that, saw that 9 is less than 10, so it checked the threat level, saw that it was 41, so stopped the bast***

Then 4 hours later it tried again. This time the last time it has done something wrong was already 10 days before, so the mod saw that 10 equals 10 and let it pass without checking any further.

You definitely need to change this setting. I have it setup to 90 in all my forums and wouldn't recommend anybody to change that number.

Quote from: IdanC on October 30, 2010, 10:06:19 AM
Threat level considered bad - 23

So you are saying that anybody with a threat level of 23 or less is a good guy and can enter your site un-checked.

You need to remember that the threat level is not a linear scale, but a logarithmic one. So somebody with a threat level of 20 is not 10 times worst that another one with a Th. L. of 10, but a lot worst than that.

To see this a little more clearly you can read what they said in Project Honey Pot when they invented this system:

http://www.projecthoneypot.org/threat_info.php

Of course it's up to you to set this number to anything you like, but I have it in my forums down to 10 in some of them and even 5 in others.

Quote from: IdanC on October 30, 2010, 10:06:19 AM
Threat level considered VERY bad - 40

So then, everybody with a Th.L. of 40 or less will see the captcha to prove they are human.

Well, remember what I said abut the TH.L. being a logarithmic scale. I consider that anybody with a Th.L. more than 30 is for sure a spammer, so I don't care if they are humans or robots. So I don't give them the chance in my forums to see the captcha and have this number down to 30 in all my forums.

Quote from: IdanC on October 30, 2010, 10:06:19 AM
Minutes to keep the data in the cache - 120

So if the mod checks an IP and finds no data about it you give that IP a "free pass" to do whatever they want in your site for 2 hours without being checked again.

I think that's too much.

The idea of the cache is to stop the logs growing too much when a robot try to see hundreds of your pages at the same time.

Remember a human has a limit in the amount of clicks per minute they can do in a web page. A human can open only a few pages per minute. But a robot hasn't got this limitation. They can open hundreds (even thousands) of pages per minute.

But remember also that the Th.L. system is also dynamic. It is changing every minute. So an IP with no data just now can have a very high Th.L. the next minute.

I have this number down to 5 minutes in my forums, but I have been thinking maybe that's even too high as well, so maybe I will change it to 1 or 2 minutes.


Quote from: IdanC on October 30, 2010, 10:06:19 AM
Hours until an IP needs to prove again it's human - 200

200 hours is a little more than 8 days.

So if somebody prove in your site to be a human, you give that IP again a "free pass" to do whatever they like in your site for more than 8 days without being checked again.

If that human guy was using a dynamic IP (as almost everybody in the world) theirs IP will be changing very often, every 2 or 3 days (sometimes dynamic IPs even change a few times on the same day) so it's almost sure that if somebody arrives to your site a few days later with that IP it will be from a completely different computer. Maybe spammer maybe not.

I have this down to 24 hours and again I think it's too high, so maybe I will put this down too in my forums.

Quote from: IdanC on October 30, 2010, 10:06:19 AM
and one last question - is there a way to clear all visitors from a certain range or geographic location?

No with this mod. You can do it with a lot more methods though. The easiest ones are using .htaccess or using SMF "Ban List".

I don't like it anyway. Having a lot of spammers from one country doesn't mean everybody in that country is a spammer.

For example almost half the spammers in my forums are from USA, almost the other half from Russia and a few more from India, Hong-Kong, Korea, etc.

But I have legal visitors, friends and even members from almost all that countries, so it will be a bad idea if I stopped everybody from any of these countries entering my sites.

==========

Of course, as I said, it's up to you what you do with your site.



==========
EDIT:
I just modified this post to sort some spelling mistakes.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

IdanC



this is by far the best answer i ever got in these forums!

a small donation shall be made soon to show my appreciation  :)

Advertisement: