Advertisement:

Author Topic: [MOD][PENDING] Spam Blocker - Anti-spam modification to restrict or limit access  (Read 89012 times)

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Spam Blocker

Developed for SMF forums c/o Underdog @ http://webdevelop.comli.com
Copyright 2013 underdog@webdevelop.comli.com
Beta testers: Skhilled & TinMan

Purpose and/or usage of this software package:

The purpose of this anti-spam modification software package is to detect unsolicited web traffic (a.k.a. Spam) and restrict and/or limit its access from registering as users and/or participating on your Simple Machines Forum website.

This software package is distributed under the terms of its Freeware License whereas all of its condtional terms are noted within its license link from your SMF Administraion Panel and/or the link provided in this paragraph. If you do not agree to the terms shown in the license, do not download and/or use this software package. 

If you commend this software package and/or any other contributions that underdog@webdevelop.comli.com develops for the SMF community, please feel free to make a donation to paypal using the image/link provided below.
Thank you for opting to use this software package.





Spam Blocker Features:

  • User IP's and/or Email's are checked externally on anti-spam source sites
  • IP's/Emails that are flagged/reported as spam can be blocked from registering on your forum
  • Flagged IP's/Emails can be banned upon registration
  • Flagged IP's/Emails can be redirected to a specific URL during the registration process (ie. Honeypot script)
  • Topics/Replies can be filtered through the Akismet database
  • Topics/Replies can be reported to the Stop Forum Spam database
  • Specific membergroup(s) and preset number of initial posts can be opted for post filtering
  • Options for specific ban restrictions
  • Custom user & error messages
  • Whitelist that allows specific IP's/Ranges to bypass the IP/Email check
  • Blacklist of IP's/Ranges added to the ban list by Spam Blocker
  • 1 hour cache of data to limit resource usage
  • Second user name link in Admin->Members now redirects to Spam Blocker IP/Email Lookup
  • License and guide for usage are provided on the Administration page



Current anti-spam resources:

Registration
  • Akismet Email Analysis
  • Stop Forum Spam Email Analysis
  • Stop Forum Spam IP Analysis
  • Project Honeypot IP Analysis
  • Spamhaus IP Analysis (sbl-xbl block list)

Posts/Topics
  • Akismet Filtering
  • Akismet Reporting
  • Stop Forum Spam Reporting



Annotations:
  • Do not edit the note text from entries added to your ban list from Spam Blocker. They are used as a reference for when this modification omits blacklist/ban list entities. That textarea input will be disabled for ban enitities added by Spam Blocker.
  • If an IP is somehow on the spam reporting source sites in error, it can be added to this modifications white list. It will not filter those whitelisted IP's during the registration process.



Changelog:

[Version 1.0]
+ Initial release
+ User IP's and/or Email Addresses can be checked & denied upon registration
+ Option to ban reported IP's and/or Email Addresses 
+ Option to redirect flagged entities to a specific URL during the registration process
+ Specific ban restrictions
+ Topics/Replies can be filtered through the Akismet database
+ Topics/replies can be reported to the Stop Forum Spam database
+ Editable message to be displayed to denied IP/Email (attempting registry)
+ Editable error log message
+ Whitelist to bypass IP check
+ Blacklist to display IP's added by Spam Blocker
+ Blacklist Optimization (Ban list comparison)
+ Option to delete expired Blacklist IP's
+ Option to allow Spamblocker to auto delete expired ban's
+ 1 hour cache (ip, time, pass/fail) to limit resource usage
+ License and guide provided on Admin page



Recommended minimal requirements:
Server:   PHP 5.2+ with libxml, cURL, socket connections & DOM enabled
              MYSQL 5.0+ using MyISAM or InnoDB engine
Browser Add-Ons (for admin): Adobe Flashplayer 11.5+, JRE 7.10+, HTML5 capability
SMF Version: 2.0.4+




Disclaimers:

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Please read all other license agreements contained within this package.



Notes:
 The simplest way to set up this modification is to acquire all the necessary API keys, enter them in the mods configuration and then execute Default Configuration from its configuration template.  After which you can tweak a few settings to your desired specifics.

  For post filtering it is suggested to go into your permission settings and Enable Post Filtering for your lowest post ranked (first) membergroup.



Reminder: Back up your database prior to installing any modification!


Files located here:GitHub Repo
Download: Spam Blocker for SMF

« Last Edit: February 09, 2014, 11:28:11 PM by -Underdog- »

Online vbgamer45

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 19,350
    • smfhacks on Facebook
    • VBGAMER45 on GitHub
    • @createaforum on Twitter
    • SMF For Free
Looks great! Good feature set. Looks like everyone is working on antispam mods now which are really needed!
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Looks great! Good feature set. Looks like everyone is working on antispam mods now which are really needed!

Thank you for the positive feedback. Please feel free to test it live for a while.
In time I will add more anti-spam features to it as I already have some in mind.



  Imo the rule based filtering fallback feature works well for ie. approx. 2 posts but I believe the default setting is 5 which can be changed. Make sure to enable that option in the initial lowest post based membergroup permission setting for it to take effect. 

  There is the odd ip/email that is not in any of the source's databases where the rule based filtering seems to thwart those 100% thus far with testing.  Approx. 99% are blocked from registering altogether and the fallback takes care of the 1%. As I said, additional features will be added (which will also address the 1%) and I will attempt options that do not involve any user input (or the requirement of removing existing inputs).  Every existing and future features will have the options for disable/enable. 

  With this mod there should be no need for CAPTCHA nor any questions for registration. Imo - all other current anti-spam mods/methods can be uninstalled and/or disabled.  This will make things very user friendly for legitimate users to undergo the registration process.
 
  The most rigorous testing I was able to perform for this modification with the resources I have available to me was a forum that gets approx. 200 - 250 registration attempts daily.  My percentages are based on the performance reflected from 3 live forums where this mod is installed and where I have Admin access (one other test site is TinMan's which I do not have Admin access but he claims the mod works 100% for him). Approx. between 2 - 3 months of testing although the rule based filtering has only been in effect for the past few weeks.  Also I have some access for post moderation on Simple Portal which allows me to see spamming behavior on a forum that gets a lot of traffic which led me to implement the rule based filtering (this mod is not installed on that forum so the spammers register & post there).     

Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 68,045
    • Arantor on GitHub
Quote
Imo - all other current anti-spam mods/methods can be uninstalled and/or disabled.

IMNSHO, I disagree. I'd rather not add multiple remote lookups to a site when I can do everything without any extra load.

I'm intrigued by what you've done, don't get me wrong, I'm not a huge fan of the styling of the admin page personally but that's just a personal opinion. My concern is that advertising it as a magic bullet is not a good idea for the hammering this will generate on lesser servers.

I'd also note that some of the other solutions do actually have other things - Forum Firewall and/or Bad Behaviour do not just do anti-spam but some other kinds of protection on top.
To assume is to hope that those who came before had the presence of mind and capacity to implement the dreams of those who would come after.

You either die a hero or live long enough to see yourself become the villain. It seems you have chosen which, and now I must do the same.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

Arantor,

Quote
IMNSHO, I disagree. I'd rather not add multiple remote lookups to a site when I can do everything without any extra load.
...
My concern is that advertising it as a magic bullet is not a good idea for the hammering this will generate on lesser servers.

  I have looked at the memory consumption on my own site compared to when I did not use the mod & I don't think it uses all that much. Although I will test it to see what the approx. avg. memory consumption  (over 24 hour spans if possible) by gathering that data. (with - without = consumption .. when the approx. amount of denied registrations is close to equal).

Quote
I'm intrigued by what you've done, don't get me wrong, I'm not a huge fan of the styling of the admin page personally but that's just a personal opinion.
  Well, you are entitled to your opinion. I think it looks fairly neat & organized especially compared to how some other mods I have seen hack the crap out of existing smf templates and make them look messy/ugly.

Quote
I'd also note that some of the other solutions do actually have other things - Forum Firewall and/or Bad Behaviour do not just do anti-spam but some other kinds of protection on top.

 
  I was more or less concerned with people attempting to simultaneously use other anti-spam mods that use the same 3rd party database resources. Even though you would think it is common sense not to. Perhaps I should have worded my statement to say that it is my opinion that other ani-spam registration methods are most likely unnecessary but it is one's option to use them in conjunction with this one where like resources are not used.
   

Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 68,045
    • Arantor on GitHub
Quote
  I have looked at the memory consumption

I wasn't interested in memory consumption; the increase will be negligible. I was specifically referring to the notion of performance. I know worrying about performance seems to be a quaint idea these days.

Doing remote lookups can be quite troublesome in performance terms. I haven't checked whether you're doing any caching or not, or it's a per request look-up.

Quote
Perhaps I should have worded my statement to say that it is my opinion that other ani-spam registration methods are most likely unnecessary

And you're entitled to your opinion too. The opinion that you're basically using a sledgehammer to crack a walnut is mine, though.
To assume is to hope that those who came before had the presence of mind and capacity to implement the dreams of those who would come after.

You either die a hero or live long enough to see yourself become the villain. It seems you have chosen which, and now I must do the same.

Offline Colin

  • Lead Developer
  • SMF Hero
  • *
  • Posts: 7,765
  • Gender: Male
  • SMF Developer
    • colinschoen on GitHub
Thanks for the work on the mod, Underdog.
"If everybody is thinking alike, then somebody is not thinking." - Gen. George S. Patton Jr.

Colin

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Changes:

! Removed auto optimization of related tables feature
+ Added 24 hour cache to limit resource usage (ip, time, pass/fail)


  Imo the mod works well. While all its resources are activated/enabled there seems to be no noticeable issue regarding lag to one's forum due to resource usage. This is based on a forum that receives 200-250 spammers/registrations per day which performs just fine with this mod installed and all anti-spam resources enabled. Imo the configuration templates are very tidy and present their settings in an organized fashion.
 
   I ask kindly for anyone interested to try the mod out & comment based on your own opinions.  It is not necessary to use all of its features as one might find just a few of them to be in need or to their liking (such as the post filtering).

Thank you.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Changes:

+ 24 hour cache changed to 1 hour cache
+ recent changes to admin source file emended
+ admin functions for both optimizing and deleting blacklist truncates spamblocker_cache table
+ cache only for fail flag when spam member id added to db (always for pass flag)

File has been updated.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

re. Spam Blocker_v1.0-RC1.1x.zip

  I've given the version(s) posted in this topic a release candidate designation.  For anyone using this, update to this version but disable the post filtering for now as its rather buggy & will cause issues (needs an overhaul).

Changes:
+ changed from separate db query for cache to SMF built in cache function (3600 seconds)

Pending:
+ major overhaul to post filtering subroutines

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
re. Spam Blocker_v1.0-RC1.3x.zip


Changes:
+ post filtering subroutine fixed/updated
+ fixed enable/disable toggle for Log Reported Topics/Posts config setting
+ arrays in main config replaced with 2 multidimensional arrays (more organized structure)
+ main config changed to switch/case logic as requested by SMF Customization Team

Offline emanuele

  • SMF Super Hero
  • *******
  • Posts: 14,156
  • Gender: Male
  • THERE'S JUST ME
Code: [Select]
"UPDATE {$db_prefix}ban_itemsNo.

Code: [Select]
"UPDATE {db_prefix}ban_itemsYes.

This is in the edit for Subs-Members.php but if there are others is the same. ;)
And obviously at that point you can remove $db_prefix from the list of global.


A bit of suggestions

Code: [Select]
$displayEnable = !empty($modSettings['spamBlocker_PostDisplay']) ? $modSettings['spamBlocker_PostDisplay'] : 2;
$maxPostcount = !empty($modSettings['spamBlocker_postCount']) ? $modSettings['spamBlocker_postCount'] : 0;
$postAkismet = !empty($modSettings['spamBlocker_akismetPost']) ? $modSettings['spamBlocker_akismetPost'] : 2;
$postSFS = !empty($modSettings['spamBlocker_PostSFS']) ? $modSettings['spamBlocker_PostSFS'] : 2;
$enableSB = !empty($modSettings['spamBlocker_enable']) ? $modSettings['spamBlocker_enable'] : 2;
$_SESSION['spamBlocker_PostCheck'] = false;
if (empty($maxPostcount))
$message['posts_spamblocker'] = -1;
if ($postAkismet + $postSFS < 4 && $enableSB == 1 && $message['can_report_spamblocker'] && $displayEnable == 1 && $message['group_spamblocker'] != 1 && $message['posts_spamblocker'] < $maxPostcount)
{
$_SESSION['spamBlocker_PostCheck'] = 'spamBlockerPostCheck';
$link = $scripturl . '?action=SpamBlockerReport;topic=' . $context['current_topic'] . ';report_spam=' . $message['id'] . ';' . $context['session_var'] . '=' . $context['session_id'];
if ($message['reported_spamblocker']['disabled'] == 'disabled')
$link = $scripturl . '?topic=' . $context['current_topic'] . '.msg' . $context['start'] . '#msg' . $message['id'];
This (in Display.template.php) and similar blocks of code (in ModerationCenter.template.php for example) should better stay in Display.php (and ModerationCenter.php) instead of in the template.

Not sure if it is worth, though:
Code: [Select]
$smcFunc['db_query']('', "INSERT INTO {db_prefix}ban_groupsUsing $smcFunc['db_insert'] is the "proper" way.
That said, since probably you are not interested in supporting any other DBMS apart from MySQL it is less relevant (but I would see a slightly more prominent notice that MySQL is the only supported ;)).

Code: [Select]
require_once($sourcedir . '/Subs.php');It shouldn't be necessary...unless you call the file from outside anything (SMF or SSI).

mb_strtolower if you want you can use $smcFunc['strtolower'] that takes care also of the remote case mb_strtolower is disabled for some reason.

cleanSpamBlockerQuery
Wouldn't be possible to do the same with just a preg_replace?
Code: [Select]
preg_replace('/[a-zA-Z0-9_\-\/\s]/', '', $word);


Take a peek at what I'm doing! ;D



Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

emanuele,

  I will take a look at your suggestions tomorrow & thank you for taking a look at this mod.

Quote from: emanuele
That said, since probably you are not interested in supporting any other DBMS apart from MySQL it is less relevant (but I would see a slightly more prominent notice that MySQL is the only supported ;)).
  One thing though, I made sure it stated quite clearly the mod's requirements during installation, in its guide and its title page.
Quote from: -Underdog-
Recommended minimal requirements:
Server:   PHP 5.2+ with libxml, cURL, socket connections & DOM enabled
              MYSQL 5.0+ using MyISAM or InnoDB engine
Browser Add-Ons (for admin): Adobe Flashplayer 11.5+, JRE 7.10+, HTML5 capability
SMF Version: 2.0.4+

... how can I be clearer than that?

Offline emanuele

  • SMF Super Hero
  • *******
  • Posts: 14,156
  • Gender: Male
  • THERE'S JUST ME
Considering I missed it twice, I'd say at least move the requirements block below the "donate button" or something like that (but that's just me, not a requirement of sort).


Take a peek at what I'm doing! ;D



Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

Offline NanoSector

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 10,471
  • Gender: Male
  • VC321xb47@aperture:~#
    • Yoshi2889 on GitHub
Hello -Underdog-, have you had time to update your mod? :)
My Mods / Mod Builder - A tool to easily create mods / Blog
"I've heard from a reliable source that the Answer is 42. But, still no word on what the question is."

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
re. Spam Blocker_v1.0-RC1.4x.zip


Changes:
! all $db_prefix changed to db_prefix (also omitted globals)
! variable assignment for display/moderation now in source files ($context)
! removed all require_once($sourcedir . '/Subs.php');  (not necessary)
! mb_strtolower changed to $smcFunc['strtolower'] for post filtering
! recommended requirements relocated for mod's readme text file
! license updated
! donation link updated

Notes:
? cleanSpamBlockerQuery -> regex not implemented as the current routine contains filter for mysql commands when necessary
?  $smcFunc['db_insert'] not implemented .. result is the same (not necessary)

Offline NanoSector

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 10,471
  • Gender: Male
  • VC321xb47@aperture:~#
    • Yoshi2889 on GitHub
?  $smcFunc['db_insert'] not implemented .. result is the same (not necessary)
You need to change this, or your mod will not be approved. db_insert is not there because you can do it with queries, you should insert stuff with it.
My Mods / Mod Builder - A tool to easily create mods / Blog
"I've heard from a reliable source that the Answer is 42. But, still no word on what the question is."

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
You need to change this, or your mod will not be approved. db_insert is not there because you can do it with queries, you should insert stuff with it.

Yoshi,

    Just to clarify.. I realize the SMF language concerning that part of the smcFunc function uses the word query, but from an actual mysql standpoint creating tables, putting data into them and retrieving data from them are all examples of queries.
ie. read the first paragraph here: Entering mysql queries. They use SELECT in their examples but all the syntax I just described is noted in the first paragraph.

  The way the original statement was worded from Emanuele, I figured it was a suggestion. Using $smcFunc['db_query'] allows me to apply any of the mysql syntax & it works just fine. Why am I being forced to use another part of the smcFunc function that essentially does the same thing?


Offline NanoSector

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 10,471
  • Gender: Male
  • VC321xb47@aperture:~#
    • Yoshi2889 on GitHub
Why am I being forced to use another part of the smcFunc function that essentially does the same thing?
Why would we create a function to insert data when you can do the same in a query?

$smcFunc['db_insert'] performs additional checks and routines on the data inserted which $smcFunc['db_query'] doesn't do.

If $smcFunc['db_insert'] didn't add anything to manually inserting data with _query then we wouldn't mind, but since it does...
My Mods / Mod Builder - A tool to easily create mods / Blog
"I've heard from a reliable source that the Answer is 42. But, still no word on what the question is."

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Why would we create a function to insert data when you can do the same in a query?

$smcFunc['db_insert'] performs additional checks and routines on the data inserted which $smcFunc['db_query'] doesn't do.

If $smcFunc['db_insert'] didn't add anything to manually inserting data with _query then we wouldn't mind, but since it does...

  Yoshi,
  I filter all the data prior to insertion, be it text or integer. I always make sure the data is safe prior to the insertion therefore I do not understand why I am being forced to change it.