Advertisement:

Author Topic: [MOD][PENDING] Spam Blocker - Anti-spam modification to restrict or limit access  (Read 88959 times)

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Spam Blocker

Developed for SMF forums c/o Underdog @ http://webdevelop.comli.com
Copyright 2013 underdog@webdevelop.comli.com
Beta testers: Skhilled & TinMan

Purpose and/or usage of this software package:

The purpose of this anti-spam modification software package is to detect unsolicited web traffic (a.k.a. Spam) and restrict and/or limit its access from registering as users and/or participating on your Simple Machines Forum website.

This software package is distributed under the terms of its Freeware License whereas all of its condtional terms are noted within its license link from your SMF Administraion Panel and/or the link provided in this paragraph. If you do not agree to the terms shown in the license, do not download and/or use this software package. 

If you commend this software package and/or any other contributions that underdog@webdevelop.comli.com develops for the SMF community, please feel free to make a donation to paypal using the image/link provided below.
Thank you for opting to use this software package.





Spam Blocker Features:

  • User IP's and/or Email's are checked externally on anti-spam source sites
  • IP's/Emails that are flagged/reported as spam can be blocked from registering on your forum
  • Flagged IP's/Emails can be banned upon registration
  • Flagged IP's/Emails can be redirected to a specific URL during the registration process (ie. Honeypot script)
  • Topics/Replies can be filtered through the Akismet database
  • Topics/Replies can be reported to the Stop Forum Spam database
  • Specific membergroup(s) and preset number of initial posts can be opted for post filtering
  • Options for specific ban restrictions
  • Custom user & error messages
  • Whitelist that allows specific IP's/Ranges to bypass the IP/Email check
  • Blacklist of IP's/Ranges added to the ban list by Spam Blocker
  • 1 hour cache of data to limit resource usage
  • Second user name link in Admin->Members now redirects to Spam Blocker IP/Email Lookup
  • License and guide for usage are provided on the Administration page



Current anti-spam resources:

Registration
  • Akismet Email Analysis
  • Stop Forum Spam Email Analysis
  • Stop Forum Spam IP Analysis
  • Project Honeypot IP Analysis
  • Spamhaus IP Analysis (sbl-xbl block list)

Posts/Topics
  • Akismet Filtering
  • Akismet Reporting
  • Stop Forum Spam Reporting



Annotations:
  • Do not edit the note text from entries added to your ban list from Spam Blocker. They are used as a reference for when this modification omits blacklist/ban list entities. That textarea input will be disabled for ban enitities added by Spam Blocker.
  • If an IP is somehow on the spam reporting source sites in error, it can be added to this modifications white list. It will not filter those whitelisted IP's during the registration process.



Changelog:

[Version 1.0]
+ Initial release
+ User IP's and/or Email Addresses can be checked & denied upon registration
+ Option to ban reported IP's and/or Email Addresses 
+ Option to redirect flagged entities to a specific URL during the registration process
+ Specific ban restrictions
+ Topics/Replies can be filtered through the Akismet database
+ Topics/replies can be reported to the Stop Forum Spam database
+ Editable message to be displayed to denied IP/Email (attempting registry)
+ Editable error log message
+ Whitelist to bypass IP check
+ Blacklist to display IP's added by Spam Blocker
+ Blacklist Optimization (Ban list comparison)
+ Option to delete expired Blacklist IP's
+ Option to allow Spamblocker to auto delete expired ban's
+ 1 hour cache (ip, time, pass/fail) to limit resource usage
+ License and guide provided on Admin page



Recommended minimal requirements:
Server:   PHP 5.2+ with libxml, cURL, socket connections & DOM enabled
              MYSQL 5.0+ using MyISAM or InnoDB engine
Browser Add-Ons (for admin): Adobe Flashplayer 11.5+, JRE 7.10+, HTML5 capability
SMF Version: 2.0.4+




Disclaimers:

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Please read all other license agreements contained within this package.



Notes:
 The simplest way to set up this modification is to acquire all the necessary API keys, enter them in the mods configuration and then execute Default Configuration from its configuration template.  After which you can tweak a few settings to your desired specifics.

  For post filtering it is suggested to go into your permission settings and Enable Post Filtering for your lowest post ranked (first) membergroup.



Reminder: Back up your database prior to installing any modification!


Files located here:GitHub Repo
Download: Spam Blocker for SMF

« Last Edit: February 09, 2014, 11:28:11 PM by -Underdog- »

Offline vbgamer45

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 19,349
    • smfhacks on Facebook
    • VBGAMER45 on GitHub
    • @createaforum on Twitter
    • SMF For Free
Looks great! Good feature set. Looks like everyone is working on antispam mods now which are really needed!
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Looks great! Good feature set. Looks like everyone is working on antispam mods now which are really needed!

Thank you for the positive feedback. Please feel free to test it live for a while.
In time I will add more anti-spam features to it as I already have some in mind.



  Imo the rule based filtering fallback feature works well for ie. approx. 2 posts but I believe the default setting is 5 which can be changed. Make sure to enable that option in the initial lowest post based membergroup permission setting for it to take effect. 

  There is the odd ip/email that is not in any of the source's databases where the rule based filtering seems to thwart those 100% thus far with testing.  Approx. 99% are blocked from registering altogether and the fallback takes care of the 1%. As I said, additional features will be added (which will also address the 1%) and I will attempt options that do not involve any user input (or the requirement of removing existing inputs).  Every existing and future features will have the options for disable/enable. 

  With this mod there should be no need for CAPTCHA nor any questions for registration. Imo - all other current anti-spam mods/methods can be uninstalled and/or disabled.  This will make things very user friendly for legitimate users to undergo the registration process.
 
  The most rigorous testing I was able to perform for this modification with the resources I have available to me was a forum that gets approx. 200 - 250 registration attempts daily.  My percentages are based on the performance reflected from 3 live forums where this mod is installed and where I have Admin access (one other test site is TinMan's which I do not have Admin access but he claims the mod works 100% for him). Approx. between 2 - 3 months of testing although the rule based filtering has only been in effect for the past few weeks.  Also I have some access for post moderation on Simple Portal which allows me to see spamming behavior on a forum that gets a lot of traffic which led me to implement the rule based filtering (this mod is not installed on that forum so the spammers register & post there).     

Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 68,041
    • Arantor on GitHub
Quote
Imo - all other current anti-spam mods/methods can be uninstalled and/or disabled.

IMNSHO, I disagree. I'd rather not add multiple remote lookups to a site when I can do everything without any extra load.

I'm intrigued by what you've done, don't get me wrong, I'm not a huge fan of the styling of the admin page personally but that's just a personal opinion. My concern is that advertising it as a magic bullet is not a good idea for the hammering this will generate on lesser servers.

I'd also note that some of the other solutions do actually have other things - Forum Firewall and/or Bad Behaviour do not just do anti-spam but some other kinds of protection on top.
To assume is to hope that those who came before had the presence of mind and capacity to implement the dreams of those who would come after.

You either die a hero or live long enough to see yourself become the villain. It seems you have chosen which, and now I must do the same.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

Arantor,

Quote
IMNSHO, I disagree. I'd rather not add multiple remote lookups to a site when I can do everything without any extra load.
...
My concern is that advertising it as a magic bullet is not a good idea for the hammering this will generate on lesser servers.

  I have looked at the memory consumption on my own site compared to when I did not use the mod & I don't think it uses all that much. Although I will test it to see what the approx. avg. memory consumption  (over 24 hour spans if possible) by gathering that data. (with - without = consumption .. when the approx. amount of denied registrations is close to equal).

Quote
I'm intrigued by what you've done, don't get me wrong, I'm not a huge fan of the styling of the admin page personally but that's just a personal opinion.
  Well, you are entitled to your opinion. I think it looks fairly neat & organized especially compared to how some other mods I have seen hack the crap out of existing smf templates and make them look messy/ugly.

Quote
I'd also note that some of the other solutions do actually have other things - Forum Firewall and/or Bad Behaviour do not just do anti-spam but some other kinds of protection on top.

 
  I was more or less concerned with people attempting to simultaneously use other anti-spam mods that use the same 3rd party database resources. Even though you would think it is common sense not to. Perhaps I should have worded my statement to say that it is my opinion that other ani-spam registration methods are most likely unnecessary but it is one's option to use them in conjunction with this one where like resources are not used.
   

Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 68,041
    • Arantor on GitHub
Quote
  I have looked at the memory consumption

I wasn't interested in memory consumption; the increase will be negligible. I was specifically referring to the notion of performance. I know worrying about performance seems to be a quaint idea these days.

Doing remote lookups can be quite troublesome in performance terms. I haven't checked whether you're doing any caching or not, or it's a per request look-up.

Quote
Perhaps I should have worded my statement to say that it is my opinion that other ani-spam registration methods are most likely unnecessary

And you're entitled to your opinion too. The opinion that you're basically using a sledgehammer to crack a walnut is mine, though.
To assume is to hope that those who came before had the presence of mind and capacity to implement the dreams of those who would come after.

You either die a hero or live long enough to see yourself become the villain. It seems you have chosen which, and now I must do the same.

Offline Colin

  • Lead Developer
  • SMF Hero
  • *
  • Posts: 7,765
  • Gender: Male
  • SMF Developer
    • colinschoen on GitHub
Thanks for the work on the mod, Underdog.
"If everybody is thinking alike, then somebody is not thinking." - Gen. George S. Patton Jr.

Colin

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Changes:

! Removed auto optimization of related tables feature
+ Added 24 hour cache to limit resource usage (ip, time, pass/fail)


  Imo the mod works well. While all its resources are activated/enabled there seems to be no noticeable issue regarding lag to one's forum due to resource usage. This is based on a forum that receives 200-250 spammers/registrations per day which performs just fine with this mod installed and all anti-spam resources enabled. Imo the configuration templates are very tidy and present their settings in an organized fashion.
 
   I ask kindly for anyone interested to try the mod out & comment based on your own opinions.  It is not necessary to use all of its features as one might find just a few of them to be in need or to their liking (such as the post filtering).

Thank you.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Changes:

+ 24 hour cache changed to 1 hour cache
+ recent changes to admin source file emended
+ admin functions for both optimizing and deleting blacklist truncates spamblocker_cache table
+ cache only for fail flag when spam member id added to db (always for pass flag)

File has been updated.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

re. Spam Blocker_v1.0-RC1.1x.zip

  I've given the version(s) posted in this topic a release candidate designation.  For anyone using this, update to this version but disable the post filtering for now as its rather buggy & will cause issues (needs an overhaul).

Changes:
+ changed from separate db query for cache to SMF built in cache function (3600 seconds)

Pending:
+ major overhaul to post filtering subroutines

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
re. Spam Blocker_v1.0-RC1.3x.zip


Changes:
+ post filtering subroutine fixed/updated
+ fixed enable/disable toggle for Log Reported Topics/Posts config setting
+ arrays in main config replaced with 2 multidimensional arrays (more organized structure)
+ main config changed to switch/case logic as requested by SMF Customization Team

Offline emanuele

  • SMF Super Hero
  • *******
  • Posts: 14,156
  • Gender: Male
  • THERE'S JUST ME
Code: [Select]
"UPDATE {$db_prefix}ban_itemsNo.

Code: [Select]
"UPDATE {db_prefix}ban_itemsYes.

This is in the edit for Subs-Members.php but if there are others is the same. ;)
And obviously at that point you can remove $db_prefix from the list of global.


A bit of suggestions

Code: [Select]
$displayEnable = !empty($modSettings['spamBlocker_PostDisplay']) ? $modSettings['spamBlocker_PostDisplay'] : 2;
$maxPostcount = !empty($modSettings['spamBlocker_postCount']) ? $modSettings['spamBlocker_postCount'] : 0;
$postAkismet = !empty($modSettings['spamBlocker_akismetPost']) ? $modSettings['spamBlocker_akismetPost'] : 2;
$postSFS = !empty($modSettings['spamBlocker_PostSFS']) ? $modSettings['spamBlocker_PostSFS'] : 2;
$enableSB = !empty($modSettings['spamBlocker_enable']) ? $modSettings['spamBlocker_enable'] : 2;
$_SESSION['spamBlocker_PostCheck'] = false;
if (empty($maxPostcount))
$message['posts_spamblocker'] = -1;
if ($postAkismet + $postSFS < 4 && $enableSB == 1 && $message['can_report_spamblocker'] && $displayEnable == 1 && $message['group_spamblocker'] != 1 && $message['posts_spamblocker'] < $maxPostcount)
{
$_SESSION['spamBlocker_PostCheck'] = 'spamBlockerPostCheck';
$link = $scripturl . '?action=SpamBlockerReport;topic=' . $context['current_topic'] . ';report_spam=' . $message['id'] . ';' . $context['session_var'] . '=' . $context['session_id'];
if ($message['reported_spamblocker']['disabled'] == 'disabled')
$link = $scripturl . '?topic=' . $context['current_topic'] . '.msg' . $context['start'] . '#msg' . $message['id'];
This (in Display.template.php) and similar blocks of code (in ModerationCenter.template.php for example) should better stay in Display.php (and ModerationCenter.php) instead of in the template.

Not sure if it is worth, though:
Code: [Select]
$smcFunc['db_query']('', "INSERT INTO {db_prefix}ban_groupsUsing $smcFunc['db_insert'] is the "proper" way.
That said, since probably you are not interested in supporting any other DBMS apart from MySQL it is less relevant (but I would see a slightly more prominent notice that MySQL is the only supported ;)).

Code: [Select]
require_once($sourcedir . '/Subs.php');It shouldn't be necessary...unless you call the file from outside anything (SMF or SSI).

mb_strtolower if you want you can use $smcFunc['strtolower'] that takes care also of the remote case mb_strtolower is disabled for some reason.

cleanSpamBlockerQuery
Wouldn't be possible to do the same with just a preg_replace?
Code: [Select]
preg_replace('/[a-zA-Z0-9_\-\/\s]/', '', $word);


Take a peek at what I'm doing! ;D



Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

emanuele,

  I will take a look at your suggestions tomorrow & thank you for taking a look at this mod.

Quote from: emanuele
That said, since probably you are not interested in supporting any other DBMS apart from MySQL it is less relevant (but I would see a slightly more prominent notice that MySQL is the only supported ;)).
  One thing though, I made sure it stated quite clearly the mod's requirements during installation, in its guide and its title page.
Quote from: -Underdog-
Recommended minimal requirements:
Server:   PHP 5.2+ with libxml, cURL, socket connections & DOM enabled
              MYSQL 5.0+ using MyISAM or InnoDB engine
Browser Add-Ons (for admin): Adobe Flashplayer 11.5+, JRE 7.10+, HTML5 capability
SMF Version: 2.0.4+

... how can I be clearer than that?

Offline emanuele

  • SMF Super Hero
  • *******
  • Posts: 14,156
  • Gender: Male
  • THERE'S JUST ME
Considering I missed it twice, I'd say at least move the requirements block below the "donate button" or something like that (but that's just me, not a requirement of sort).


Take a peek at what I'm doing! ;D



Hai bisogno di supporto in Italiano?

Aiutateci ad aiutarvi: spiegate bene il vostro problema: no, "non funziona" non è una spiegazione!!
1) Cosa fai,
2) cosa ti aspetti,
3) cosa ottieni.

Offline NanoSector

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 10,471
  • Gender: Male
  • VC321xb47@aperture:~#
    • Yoshi2889 on GitHub
Hello -Underdog-, have you had time to update your mod? :)
My Mods / Mod Builder - A tool to easily create mods / Blog
"I've heard from a reliable source that the Answer is 42. But, still no word on what the question is."

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
re. Spam Blocker_v1.0-RC1.4x.zip


Changes:
! all $db_prefix changed to db_prefix (also omitted globals)
! variable assignment for display/moderation now in source files ($context)
! removed all require_once($sourcedir . '/Subs.php');  (not necessary)
! mb_strtolower changed to $smcFunc['strtolower'] for post filtering
! recommended requirements relocated for mod's readme text file
! license updated
! donation link updated

Notes:
? cleanSpamBlockerQuery -> regex not implemented as the current routine contains filter for mysql commands when necessary
?  $smcFunc['db_insert'] not implemented .. result is the same (not necessary)

Offline NanoSector

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 10,471
  • Gender: Male
  • VC321xb47@aperture:~#
    • Yoshi2889 on GitHub
?  $smcFunc['db_insert'] not implemented .. result is the same (not necessary)
You need to change this, or your mod will not be approved. db_insert is not there because you can do it with queries, you should insert stuff with it.
My Mods / Mod Builder - A tool to easily create mods / Blog
"I've heard from a reliable source that the Answer is 42. But, still no word on what the question is."

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
You need to change this, or your mod will not be approved. db_insert is not there because you can do it with queries, you should insert stuff with it.

Yoshi,

    Just to clarify.. I realize the SMF language concerning that part of the smcFunc function uses the word query, but from an actual mysql standpoint creating tables, putting data into them and retrieving data from them are all examples of queries.
ie. read the first paragraph here: Entering mysql queries. They use SELECT in their examples but all the syntax I just described is noted in the first paragraph.

  The way the original statement was worded from Emanuele, I figured it was a suggestion. Using $smcFunc['db_query'] allows me to apply any of the mysql syntax & it works just fine. Why am I being forced to use another part of the smcFunc function that essentially does the same thing?


Offline NanoSector

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 10,471
  • Gender: Male
  • VC321xb47@aperture:~#
    • Yoshi2889 on GitHub
Why am I being forced to use another part of the smcFunc function that essentially does the same thing?
Why would we create a function to insert data when you can do the same in a query?

$smcFunc['db_insert'] performs additional checks and routines on the data inserted which $smcFunc['db_query'] doesn't do.

If $smcFunc['db_insert'] didn't add anything to manually inserting data with _query then we wouldn't mind, but since it does...
My Mods / Mod Builder - A tool to easily create mods / Blog
"I've heard from a reliable source that the Answer is 42. But, still no word on what the question is."

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Why would we create a function to insert data when you can do the same in a query?

$smcFunc['db_insert'] performs additional checks and routines on the data inserted which $smcFunc['db_query'] doesn't do.

If $smcFunc['db_insert'] didn't add anything to manually inserting data with _query then we wouldn't mind, but since it does...

  Yoshi,
  I filter all the data prior to insertion, be it text or integer. I always make sure the data is safe prior to the insertion therefore I do not understand why I am being forced to change it.

Offline NanoSector

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 10,471
  • Gender: Male
  • VC321xb47@aperture:~#
    • Yoshi2889 on GitHub
Why would we create a function to insert data when you can do the same in a query?

$smcFunc['db_insert'] performs additional checks and routines on the data inserted which $smcFunc['db_query'] doesn't do.

If $smcFunc['db_insert'] didn't add anything to manually inserting data with _query then we wouldn't mind, but since it does...

  Yoshi,
  I filter all the data prior to insertion, be it text or integer. I always make sure the data is safe prior to the insertion therefore I do not understand why I am being forced to change it.
Even though you sanitise the data you still need to use db_insert, end of discussion. Change it if you want your mod approved.
My Mods / Mod Builder - A tool to easily create mods / Blog
"I've heard from a reliable source that the Answer is 42. But, still no word on what the question is."

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Even though you sanitise the data you still need to use db_insert, end of discussion. Change it if you want your mod approved.

  Sadly, your mannerism does not surprise me.



re. Spam Blocker_v1.0-RC1.5x.zip

Changes:
! $smcFunc['db_insert'] implemented

Offline Colin

  • Lead Developer
  • SMF Hero
  • *
  • Posts: 7,765
  • Gender: Male
  • SMF Developer
    • colinschoen on GitHub
Hi Underdog,

I can see your frustration, but there is a reason for standardization. Let me try to clear it up, if I may. If for example, SMF changes the method of sanitation of data down the road, we would want it to apply to all modifications. Your mod does mimic the function now, but the functions are evolving and we want your mod to be used in the community for as long as possible. Thank you so much for your understanding.
"If everybody is thinking alike, then somebody is not thinking." - Gen. George S. Patton Jr.

Colin

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

Colin,

  Now I understand the reasoning perfectly & I thank you for that explanation.

Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 68,041
    • Arantor on GitHub
Actually, it's nothing whatsoever to do with sanitisation, since the same sanitisation protection is really employed at the query level anyway through the parameterisation system.

db_insert does a bunch of things that db_query won't do for inserting on non-MySQL platforms. It does all kinds of handling around the primary and foreign keys on both PostgreSQL and SQLite, and there's a bunch of stuff being mimicked for getting the last inserted id.
To assume is to hope that those who came before had the presence of mind and capacity to implement the dreams of those who would come after.

You either die a hero or live long enough to see yourself become the villain. It seems you have chosen which, and now I must do the same.

Offline phill104

  • Semi-Newbie
  • *
  • Posts: 13
Underdog,

Many thanks for the mod, it seems to be working well for us. One thing I feel that could improve it is the facility to check members that are already registered. I am aware you can manually enter an IP and an Email address in the box but that is time consuming to copy the details from the members page into the box on the mod. Would it be possible to add a button on the members page that jumps straight to the manual entry page of your mod with the users details already filled out?

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Arantor,

  Yes I am aware it was asked so that it will be compatible with various database types although the mod does state that the recommended requirement is MYSQL.  It is my experience that the $smcFunc['db_insert'] function does not seem to allow me to change just 1 or a few columns of data whereas I seem to be forced to reading a whole row of data, make the necessary changes and then use the db_insert function to replace that row. Am I wrong here or does the current smcFunc functions not include an equivalent for UPDATE (single+ columns .. less than all of them) that works for various db syntax?

phill104,

  Yes I will add that option and it is a very good idea.  Thank you for showing interest in the mod.

Regards.



Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 68,041
    • Arantor on GitHub
No, $smcFunc['db_query'] should be used for UPDATE for which a simple function can't be made. If you'd like to see what a db_update function would look like, I invite you to the joy farm that is Ruby On Rails.

db_insert is for inserting new rows or completely replacing rows, seeing how it can do either INSERT or REPLACE queries (or, in some backends' case, DELETE followed by INSERT)

If you're adding new rows or completely replacing existing rows based solely on their primary key, db_insert should be used, especially as you can add rows en masse.

If you're not adding complete rows, db_query is the way to go. Remember that an insert can work without supplying every column if appropriate defaults are used.
To assume is to hope that those who came before had the presence of mind and capacity to implement the dreams of those who would come after.

You either die a hero or live long enough to see yourself become the villain. It seems you have chosen which, and now I must do the same.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Quote from: Arantor
If you're adding new rows or completely replacing existing rows based solely on their primary key, db_insert should be used, especially as you can add rows en masse.

  Yes, a primary key or composite keys. I am aware of this but what I am asking is can I use UPDATE in a query and have it compatible with the various db types or should I be changing any instances of UPDATE to smcFunc db_insert to make it compatible? I am very familiar with MYSQL but have no experience with other database types at this time to know if the syntax is the same across the board.

 

Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 68,041
    • Arantor on GitHub
Yes, you should be able to use UPDATE for other backends via db_query as normal. The syntax should be comparable enough, just provided you don't do multi-table updates (MySQL can sometimes do that, the others can't)
To assume is to hope that those who came before had the presence of mind and capacity to implement the dreams of those who would come after.

You either die a hero or live long enough to see yourself become the villain. It seems you have chosen which, and now I must do the same.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

  On that note, I have no idea why they are neglecting to accept this modification as I implemented the mandatory changes that were asked. Oh well, let it collect dust in this board.
 

Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 68,041
    • Arantor on GitHub
Because you generally have to send a PM saying you've fixed everything then it goes back in the queue. Then someone has to review it etc. There was a time you'd usually get same day service.
To assume is to hope that those who came before had the presence of mind and capacity to implement the dreams of those who would come after.

You either die a hero or live long enough to see yourself become the villain. It seems you have chosen which, and now I must do the same.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

  I did that back in June and have not heard anything since. I did update it a few times since but I'm told I can do that whilst its in the queue and nobody looked at any of those files. I sent another PM just a few minutes ago and will await a response. Someone downloaded from the dl site after our conversation here so perhaps it is now being looked at.
 

Offline Suki

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 15,084
  • Kaizoku Jotei
    • MissAllSunday on GitHub
    • SMF mods
Just a few suggestions:

When searching for text as reference, try not to search for the entire line with tabs at the beginning, this search for example:

<![CDATA[               'notes' => $row['notes'],]]>

Can be done the same way without the tabs at the beginning:

<![CDATA['notes' => $row['notes'],]]>

You could use sprintf instead of str_replace: PHP String Replacement Speed Comparison

Not really mandatory but would be nice if your queries would be formatted to increase readability. Also, would be good to use the casting array and to check for every var existence before passing it to the query. Yes, most of the queries here have values that doesn't have to be empty but still, better safe than sorry.

Prob not a good idea to expand compatibility to, 2.99.99 its is better to keep it down to 2.0.99, 2.1 is still to young and could still have a bunch of changes, specially in templates, plus, we don't even know how SMF 2.95 would look like!

Suggestion, keep the same folder format SMF uses, a sources folder and a Themes folder, that way you only need to move both folders on install and helps things to be more organized specially for big mods with lots of files such as this one.

Watch your tabs and spaces on empty lines and at the end of text. Things like

echo '-->-->-->-->-->                               
--->-->--><a id="spamLink_' . $item['id']

Would be printed as it is by PHP.
Look at them. They're just asking for it. Maybe the human race deserves to be wiped out.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
Suki,

  I appreciate the suggestions but some of what you say is a matter of personal opinion. 

  • When searching for a line of code I like to include the preceding tabs (the whole line if you will) and I will always continue to do that.
  • str_replace is still one of the fastest methods
  • The db queries are tabbed out nicely and I do not understand the issue you speak of. Atm the queries you seem to be speaking about are inserts from SpamBlocker.php? It first inserts some data into an auto increment table (ban_groups) and then reads the id of that insertion prior to writing to the ban_items table. Since both normally work in unison, I deemed that unnecessary for the ban_items table although it could be implemented and then use conditional logic to opt insert or replace. As for the spamblocker_blacklist table it attempts a deletion first which was actually a shorter coded preliminary query than what you speak of & would also work for the ban_items table if I add that.
  • 2.09.99 .. I agree with that and already thought of it prior but neglected to change it.  I think something was changed regarding the hooks in 2.1 and you are correct this won't work with it as I believe I tested that some time ago. Still, this version description claims SMF 2.0.X compatibility and what is described to this mod's users is correct.
  • Themes and Sources folders in the installer... ya something I thought of prior also and did not do yet although I will change that.
  • I don't know what you are referring to regarding tabs and spaces. I use Komodo for files as my editor which is a proper code editor (has settings for each file type). It is set to 8 for the tabbing (4 or 8 is the norm & up to preference).. I can not see any issues regarding tabbing, to what are you referring?

Regards.


Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 68,041
    • Arantor on GitHub
Quote
When searching for a line of code I like to include the preceding tabs (the whole line if you will) and I will always continue to do that.

This is largely a matter of style. There are times it can make a difference. I personally wouldn't care about it (and I used to be the most of a hard-ass about modifications)

Quote
str_replace is still one of the fastest methods

In the uses given, sprintf is even faster.

Quote
I don't know what you are referring to regarding tabs and spaces. I use Komodo for files as my editor which is a proper code editor (has settings for each file type). It is set to 8 for the tabbing (4 or 8 is the norm & up to preference).. I can not see any issues regarding tabbing, to what are you referring?

The issue being referred to is that some lines, notably inside echos, have trailing spaces/tabs after the echo statement.

The example given:
echo '-->-->-->-->-->                               
--->-->--><a id="spamLink_' . $item['id']

There are multiple tabs in the first line of that which are still output by PHP, but shouldn't really be there. It's more of a code cleanliness thing not to mention a little bandwidth saving.
To assume is to hope that those who came before had the presence of mind and capacity to implement the dreams of those who would come after.

You either die a hero or live long enough to see yourself become the villain. It seems you have chosen which, and now I must do the same.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

  The actual html is tabbed in such a way to be lined up properly. Each echo drops down a line after a single quote to that effect and then is tabbed out to line up with the code that precedes it. If a tag is open ie. a div, span hyperlink, etc. , the code between the open & close of those tags is tabbed to show it is within it. When you peruse a file it all lines up properly as it should.

Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 68,041
    • Arantor on GitHub
Um... there are extra spaces BEFORE the line break...
To assume is to hope that those who came before had the presence of mind and capacity to implement the dreams of those who would come after.

You either die a hero or live long enough to see yourself become the villain. It seems you have chosen which, and now I must do the same.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
  You respond faster than I can come back with something else.. geesh  :o .. If I understand you correctly, I can simply go through every file with Komodo and use the remove trailing whitespace command.

Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 68,041
    • Arantor on GitHub
Yes, that's exactly what we're suggesting...
To assume is to hope that those who came before had the presence of mind and capacity to implement the dreams of those who would come after.

You either die a hero or live long enough to see yourself become the villain. It seems you have chosen which, and now I must do the same.

Offline Suki

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 15,084
  • Kaizoku Jotei
    • MissAllSunday on GitHub
    • SMF mods
Yes, most of it are personal preferences, personal preferences that helps reducing support requests.
Quote
When searching for a line of code I like to include the preceding tabs (the whole line if you will) and I will always continue to do that.

Until another mod modifies the exact same line before yours and replaces the tabs with dots or simply doesn't put the tabs back or even the exact number of tabs, making your edit to fail and naturally, the user will blame you, not the other mod author.

It is a really simple change and believe me, it will save you a few headaches when giving support.

Quote
but neglected to change it.  I think something was changed regarding the hooks in 2.1 and you are correct this won't work with it as I believe I tested that some time ago. Still, this version description claims SMF 2.0.X compatibility and what is described to this mod's users is correct.

What it claims to do and what it actually does are different things, over performing to what was claimed is just as bad as been unable to perform what is claimed. This comes from experience dealing with support, users generally don't wait until mod authors adjust their mods to be compatible with the new version, they just install them, it is preferable to not be able to install the mod (in a normal way) than give them the ability to install a mod that just won't work with their version.

Again, a simple change can made the difference between you been happily enjoying a cold one and watching the sunset or been fed up and angry in front of your computer, choice is yours of course, I'm just offering my experience and nothing more.

Quote
The db queries are tabbed out nicely and I do not understand the issue you speak of.

Subs-SpamBlocker, Subs-SpamBlockerAdmin, SpamBlockerAdmin  has queries on a single line and none of them uses the casting array.  Sometime ago casting all values was mandatory, I don't know whats the policy these days.

Regardless of personal preferences, code should be easy to read by humans, after all, thats the primary role for all languages.

Your license doesn't allow re-distribution, can you please add a simple note in the mod's page indicating you give SMF and SM  written consent to re-distribute your work.
Look at them. They're just asking for it. Maybe the human race deserves to be wiped out.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

Suki,

  I take everything you said into consideration and will make the changes later tonight. May I also add that I realize you were correct about the tabbing of the MYSQL whereas I do not know why I did not do it properly before. I suppose as I do more and more projects my coding habits improve and atm tabbing things properly comes natural. Thank you again for looking at the mod and giving me the feedback I need.

Arantor,

  I also thank you for your feedback and experience.

Regards.

 

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

Mod has been updated.

Spam Blocker_v1.0-RC1.7

Changes:
+ added link in Admin->Members to redirect to Lookup template
! fixed source code for post filtering
! adjusted copyright to allow Simple Machines to distribute the modification from SMF download site
! fixed tabs for all mysql queries
! fixed install to only work for SMF 2.0.X series
! reorganized install files into appropriate folders (same as SMF defaults)
! used casting array in smcFunc functions where applicable
! removed all empty whitespaces at end of lines in every file



Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

I fixed the ones I missed in the admin sources file with RC1.71 ... my apologies.

Offline phill104

  • Semi-Newbie
  • *
  • Posts: 13
Many thanks for the update and feature inclusion. Spam is an ever increasing problem and while there are many great tools out there some just don't hit the mark. Some work but require far too much admin intervention and when you have a big forum like ours that at times is getting 500+ spam registrations per day the work load is just silly. This seems to work well and hopefully keeps the users happy by not having to jump through hoops to simply register.

Offline Suki

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 15,084
  • Kaizoku Jotei
    • MissAllSunday on GitHub
    • SMF mods

Mod has been updated.

Spam Blocker_v1.0-RC1.7

Changes:
+ added link in Admin->Members to redirect to Lookup template
! fixed source code for post filtering
! adjusted copyright to allow Simple Machines to distribute the modification from SMF download site
! fixed tabs for all mysql queries
! fixed install to only work for SMF 2.0.X series
! reorganized install files into appropriate folders (same as SMF defaults)
! used casting array in smcFunc functions where applicable
! removed all empty whitespaces at end of lines in every file




Thnks :)

On my part theres nothing more to edit or change, there might be bugs but thats only natural and they will only arise after some production usage. Guess just have to wait for the cust team. Good luck :)
Look at them. They're just asking for it. Maybe the human race deserves to be wiped out.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca

Suki,

  Thank you for unveiling your expertise.



  I added a blacklist link as one of the admin nav button children for the RC1.71 but didn't realize it loads my language file after that particular hook. I edited in a load language file command for that hook to fix that issue in RC1.72 (and for the mod site 1.0 copy).

Offline phill104

  • Semi-Newbie
  • *
  • Posts: 13
How are you getting on with this mod? We've been using it for a while and it ended up overloading our DB as the ban list ended up so large queries to it were taking so long. We emptied the ban list and stopped the mod from adding to it in the short term.

Offline Chen Zhen

  • Sophist Member
  • *****
  • Posts: 1,028
  • Gender: Male
  • If you're going through hell, keep going!
    • Underdog-01 on GitHub
    • WebDev.ca
phill104,

  Although it allows an option to permanently ban any flagged IP's, it is best to set it to ie. a 1, 2 or 3 day ban whereas it will delete ban entries it added within that time frame.  You can also leave it set to not ban an ip address as you have stated which should work just fine as it cache's flagged ip's for 1 hour.
 
Regards.