News:

Wondering if this will always be free?  See why free is better.

Main Menu

Stop Spammer

Started by M-DVD, December 31, 2008, 07:31:43 AM

Previous topic - Next topic

Angie on Dialysis

#1040
Quote from: snoopy_virtual on January 03, 2011, 08:00:56 PM
@Angie KidneyKorner

I suppose you don't have mod httpBL. If you had it you wouldn't have so many spammers registering.

@plugger

I wouldn't recommend you to check usernames. A lot of normal names are already inside the DB, so it's not really worth it. I have been even thinking on deleting completely that option.

I actually do have httpBL on my medical forums



but Sandmansa doesn't have it on his and Plugger doesn't have it on his. I help both of them out when it comes to educating them and beating the spammers. Same with yet a 3rd guy who isn't a member of these support forums who I convinced to switch from phpBB3 to SMF.

@ Plugger: Are you seeing the magnifying images yet?

Because for me as moderator on your forums with the power to see the members list from admin I don't..
And I checked on my own forums and on Sandmansa (where I have admin with him) and we both have it there.

It is the admin section where you Manage members.

Because you have no pending spammers right now I just looked at all members and there should be magnifying glasses there as well and there isn't..
...index.php?action=viewmembers;sa=all

I will take a picture the next time a spammer is flagged on your forums Plugger to post here as I don't think they realize what your issue is.

All I can do is show how it USED to look on his forums .. don't know what happened to break it:

How it SHOULD look:


How it looks NOW:
(best I could do since no spammers right now awaiting action)


Quote from: snoopy_virtual on January 04, 2011, 08:22:10 PM
It's strange. I had to check today a few members I had no errors. Maybe the server was down just when you were checking.

Are you still getting yellow icons?

I get that sometimes too. I just assume that at the time of it checking that it just could not reach Stop Forum Spam for some reason. Either server side where my site is hosted or Stop Forum Spam's side .. which is where Stop Spammer checks..



Sandmansa has gotten the same thing:


cypdevil

Quote from: Angie KidneyKorner on January 03, 2011, 07:19:08 AM
I don't think all of those triggers would be good indicators of a new registrant being a spammer.

I agree with you that taking just one or two of the factors would be meaningless. However, perhaps you haven't cottoned on to the statistical probability of ALL of them adding up sufficiently to give a warning or filtering out altogether. This is a Bayesian technique, the same as is used for the more sophisticated e-mail spam filtering software, such as POPFile. I don't deny the possibility of small numbers of false positives or false negatives, especially as there are not many factors to go on. However, if it is done correctly, I wouldn't mind betting it would capture 80-90% of the non-blacklisted ones that complete registration, whether activated or not. In fact, I think it would do better than the blacklist.

The beauty of the system is flexibility due to weighting and this would be easy to modify as spambotters improve their techniques to keep up with them.

As it is analogous, let's go back to e-mail spam filtering. The first apps relied on black/whitelisting and I found them pretty useless, rarely catching more than 60-70% of spammers. I then went on to POPFile, using Bayesian techniques. Unfortunately, my ISP pre-filters spam, which scares the proverbial out of me (this will be changed shortly), so that it is not given a full chance to work properly. Have a look at this:
Classification Accuracy
Messages classified:    13,455
Classification errors:    72
Accuracy:    99.46%
Because of the pre-filtering, there is a spam rate of only 8.32% of all messages. This app filters not only spam from ham, but it classifies the ham into any number of separate "buckets" (mailboxes). These are my results:
Messages Classified
Bucket         Classification Count    False Positives    False Negatives
comm         554 (4.11%)    9    5
cypnat         149 (1.10%)    0    12
ipc         8,087 (60.10%)    3    12
mug         1,753 (13.02%)    6    1
personal         1,751 (13.01%)    39    17
spam         1,120 (8.32%)    11    25
unclassified         41 (0.30%)    4

These figures apply over 16 months of received e-mails. Before my ISP applied pre-filtering my accuracy figures were usually between 99.7% and 99.9%. This was because the relatively small numbers of spam (most people get >70%) did not allow a large spam corpus to be developed.


snoopy_virtual

@cypdevil

That's why I said that anyway I thought the idea was interesting and I will think more about it.

In fact we have already been following a similar approach for other work we are working on. (You may already know about it. If not, you will vey soon). But just now I wouldn't follow that path with mod Stop Spammer.

We could, if we wanted to join all the anti-spam solutions together in just one big one, taking all the possible anti-spam measures at the same time, and with a huge config page where users would be able to switch ON and OFF any of them they wanted.

But I don't think this will be practical. Just now I am concentrating more on doing a different plugin for every different anti-spam approach but making all of them compatible with each other, so the users can decide what plugins suit more their particular forum and just install the ones they need.

Following this path of a different plugin for every different anti-spam solution, your idea would have nothing to do with mod Stop Spammer, as it's only based on Stop Forum Spam DB. Your idea would have more to do with what something like "Bad behavior" should be. Or at least that's the path I started planning when Butchs asked me if I wanted to implement bad-behavior into a SMF plugin. At the end I never had time to continue that work with Butchs and I'm not sure if he has implemented in his mod that ideas (I didn't even had time yet to really check what Butchs' Bad behavior mod actually does).

But anyway all this will change completely when we finish the other work I told you we are working on just now. So we will see what the future brings.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

snoopy_virtual

Quote from: Angie KidneyKorner on January 04, 2011, 11:30:55 PM
All I can do is show how it USED to look on his forums .. don't know what happened to break it:

Do you mean mod Stop Spammer is not working properly for you just now? I'm lost now. I thought that was already sorted.

If it's not, contact me through Skype so I can understand it better and sort it.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

Angie on Dialysis

#1044
Quote from: snoopy_virtual on January 05, 2011, 09:14:50 AM
Quote from: Angie KidneyKorner on January 04, 2011, 11:30:55 PM
All I can do is show how it USED to look on his forums .. don't know what happened to break it:

Do you mean mod Stop Spammer is not working properly for you just now? I'm lost now. I thought that was already sorted.

If it's not, contact me through Skype so I can understand it better and sort it.

For just Plugger, it is not working on his forums for both of us. It works great on my forums and on my fiance's. It is only Plugger's that isn't working.



@ cypdevil: I just meant if it was part of Stop Spammer. Sorry if I came across too harsh but I thought you meant changing Stop Spammer which I worried would have too many false positives on certain types of forums. But I do like the idea and I like Snoopy's idea of how it would be separate but compatible and think it would work great. I just was worried about it being too harsh for some forums (like gamer forums have different types of registering members normally that are not spammers than medical forums I have noticed). Flexibility is important to me as not all sites need the same type of security / flags.

@ Snoopy I like your idea for "join all the anti-spam solutions together in just one big one / different plugin for every different anti-spam approach but making all of them compatible with each other, so the users can decide what plugins suit more their particular forum and just install the ones they need.". I like the way you think.

tm82

I looks like what's happening is, once a user is flagged with 'yellow' icons, they stay 'yellow'.  I've seen new registrants get marked 'red', but when I go to check existing "Awaiting Approval" with yellow icons they stay yellow.  They got yellow originally because apparently the sfs database was down for a bit.

Does this make sense?  Any way to clear out the 'yellow' status?

snoopy_virtual

When somebody gets a yellow icon is because your server in not connecting with sfs server.

But if it's yellow now and you check the same guy again it will turn to red or blue (depending in if it's on the DB or not). It will only stay yellow if there is no connection again with sfs server.

At least that's the way the mod works for everybody. If it's not working like that for you there is a mistake somewhere.

Can any of you contact me through Skype so I can see the problem with my own eyes and sort it?

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

tm82

Quote from: snoopy_virtual on January 05, 2011, 06:28:28 PM
When somebody gets a yellow icon is because your server in not connecting with sfs server.

But if it's yellow now and you check the same guy again it will turn to red or blue (depending in if it's on the DB or not). It will only stay yellow if there is no connection again with sfs server.

At least that's the way the mod works for everybody. If it's not working like that for you there is a mistake somewhere.

Can any of you contact me through Skype so I can see the problem with my own eyes and sort it?

Sure we can connect via video on Skype.  I'll send you my skype id via email.

smartdeviceresource

@snoopy please don't remove username check as I find this much more useful than IP Address.  Almost all Spammers I've run across use checkable usernames and Emails.  IP Addresses tend to end up forbidding many of my non-spammer members.

:)
after some thought, i've learned to be humble and leave the answering to the experts, until i too am an expert.  sorry to anyone who I've confused, with my answers

snoopy_virtual

@tm82> email answered

@smartdeviceresource> Don't worry, I won't. Even if I find a better solution to do the same I will leave the options there just in case somebody prefer to do it the old way.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

tm82

@snoop-

This is in regards to existing members staying "yellow", yet new spammers trying to register are marked "red"...  In looking through code and my database, tell me if this is possible: I think it's a memory and/or PHP limitation.  I say this because when I first installed the mod, I thought, "wow seems to be a helluva lot of spammers in my membership".  So I raised the max number of members shown on a page to 900 and started going through the membership page by page scanning everyone.  Eventually it stopped working and I thought maybe stopforumspam was able to limit my connections.. ok.  But next day I noticed the problem described in my original post.  Now I'm thinking that when StopSpammer.php generates the $members_data array, this thing is huge because in looking at my database, there are thousands of entries in there where 'is_spammer' is non-zero.  So I think it's blowing up somehow... make sense?  If you agree with my analysis then I think the thing for me to do is go into the database and clear the is_spammer fields and start over.... what do you think?

snoopy_virtual

Depending on your server configuration you can have problems when trying to check too many members at the same time. I have never tried in my server to check more than 20 members at the same time, but of course the limit will be different for every server.

Anyway I think it will be faster for you to check just the "yellow" ones one at a time. If you change the value of "is_spammer" in all the DB and start all over again you will have the same problem.

If you have too many "yellow" ones maybe you can try to check them 5 at a time (to save time) but, as I said, it would be better if you check them one by one. At least, if you only check one member, you shouldn't have any problems.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

akc42

I just found a case where the unapproved member count gets to be wrong.

This is because SMF (using 2.0RC4) doesn't adjust the count if the registration method is not = 2 (admin approval required), where as StopSpammer creates unapproved members if they are a spammer.

I think the relevant code is in Subs.php and should be as follows (the approveAccountDeletion test is a patch to fix another problem with the approved membership counting).


// Are we using registration approval?
if ((!empty($modSettings['registration_method']) && $modSettings['registration_method'] == 2) || !empty($modSettings['approveAccountDeletion']) || ($modSettings['stopspammer_enable'] && $modSettings['stopspammer_api_key'] != ''))




I am not 100% sure I got the right place, because it looked to me before as through the count got updated when deleting via the approval list.  This time I deleted the members from the membership list.




Sir_Trout

Just installed this mod on Ver 1.1.12 and installed just fine with no errors. 

Then I checked if it connected to the remote host and it did.  "Your host can make remote connection with the DB"

The I went to "Awaiting Approval" to check out some new members and I can't see the mod?

I went back to "Admin > Members > Registration > Setting" and the "Enable/Disable MOD Stop Spammer:" box was not check so I check and saved it.   But when I come back it is still uncheck and I can see the mod any where?

And Ideas on how to fix this?

Thank You.

-ST

snoopy_virtual

I am finishing a new version.

It will be ready very soon.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

snoopy_virtual

Link to mod

MOD Stop Spammer v2.3.9
==================


Authors:M-DVD and snoopy_virtual
Version:2.3.9
Release:9th January 2010
Compatible With:SMF 1.1.1 - 1.1.12
SMF 2 RC4

Change Log:




Quote


Version 2.3.9 - 9th January 2010

  *Changes in the API key. From this version the API key is ONLY needed
to report spammers, NOT to check them
  !Fixed the bug about once a member is marked "yellow" always stay yellow,
reported by tm82 here with the solution proposed by himself
  +Added the buttons to "Check" and "Report" members also to the pages
"Waiting for approval" and "Waiting for activation"
  +Added a system to check if the mod is ON, if it is up-to-date and if the
connexion with Stop Forum Spam's DB is OK
  *Changed 2 lines and added 10 new lines in language files
  ?Uninstall old version and install new one
  ?No need to edit manually any template file unless you are not using
the default Theme and you have the files Register.template.php or
ManageMembers.template.php inside the folder of the particular
Theme you are using

QuoteLegend

oFeature
*Change
+Added
-Eliminated
!Bug Fixed
?How to Update

Let me explain a little some of the changes:

The first thing I want to say is this is not the big update I was planning and I have been working on lately. I have released this small update because there were a couple of things that were urgent:


  • Changes in the API key. From this version the API key is ONLY needed to report spammers, NOT to check them

When the admin in SFS (Stop Forum Spam) site asked me to remove the default API key from the mod, and ask every body using this mod to ask for their own API keys (what I did in version 2.3.7 a couple of weeks ago) I understood it wrong.

I thought I was suppose to change the mod so it would not work at all without an API key. But talking lately with SFS admin he explained to me that it's not necessary to have an API key to check his DB. You only need one if you want to report spammers. So I have changed again the mod so it works now like that.


  • Fixed the bug about once a member is marked "yellow" always stay yellow, reported by tm82 here with the solution proposed by himself

The first time they asked me about this problem I thought the mod was not working properly for them because there was a mistake in their server or something like that, but tm82 has discovered it was actually a bug in the code. Luckily he is good with code and he found the solution at the same time. Thanks mate.  ;)

===============

Apart from these 2 urgent things, there was also a few other things that I had already done for the new version (the big one I am still working on) so I have added them to this small update:


  • Added the buttons to "Check" and "Report" members also to the pages "Waiting for approval" and "Waiting for activation"

This has been requested many times and it has been in my TODO list for too long. At last it's done.


  • Added a system to check if the mod is ON, if it is up-to-date and if the connexion with Stop Forum Spam's DB is OK

Self-explanatory.

===============


  • To update: Uninstall the old version and install the new one

As always. I just put it here just in case somebody doesn't know it.

No need to edit manually any template file unless you are not using the default Theme and you have the files Register.template.php or ManageMembers.template.php inside the folder of the particular Theme you are using

===============

I will continue working on the rest of the things I have in my list. For example the bug about unapproved member count going wrong:

Quote from: akc42 on January 07, 2011, 01:30:44 PM
I just found a case where the unapproved member count gets to be wrong.

This is because SMF (using 2.0RC4) doesn't adjust the count if the registration method is not = 2 (admin approval required), where as StopSpammer creates unapproved members if they are a spammer.

... etc

Yes, the place in Subs.php you have suggested is one of the pieces of code involve, but there are more. Also the problem with that is SMF 1.x works different than 2.x when counting them, so there are a lot more things to consider there if I want the mod to work fine for both.

With the new big update, one of the things I am doing is to separate completely the files for both versions 1.x and 2.x (as I have them for example in mod httpBL) so it will be easier then to sort things like that.

El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

Sir_Trout

Quote from: Sir_Trout on January 09, 2011, 12:56:56 PM
Just installed this mod on Ver 1.1.12 and installed just fine with no errors. 

Then I checked if it connected to the remote host and it did.  "Your host can make remote connection with the DB"

The I went to "Awaiting Approval" to check out some new members and I can't see the mod?

I went back to "Admin > Members > Registration > Setting" and the "Enable/Disable MOD Stop Spammer:" box was not check so I check and saved it.   But when I come back it is still uncheck and I can see the mod any where?

And Ideas on how to fix this?

Thank You.

-ST


The new Version ficed the problem I was able to remove 98 spammers from my forum.  Many of them I have blocked myself but some I missed.

Thanks

busterone


snoopy_virtual


El verdadero sabio es aquel que lo ve todo, lo estudia todo, lo analiza todo y molesta poco.
A true wise man is he who sees everything, studies everything, analyses everything and hardly ever annoys.

henrik1782

Hi Snoopy...

This seems to be a great mod to. Just installed it.

Is it right that there is no log or place to manage this mod...? When the status box shows that it have found 3 spammers are there any way to se their IP's.

Best regards
Henrik Poulsen
My favorite mods: Forum Firewall, httBL, MessagePreviewOnHover, BoardHover Mod, VB Style Board Index, Separate Replies and Views Column, Realtime clock by Joker, ENotify, Topic Solved.

Advertisement: