[TIP/TRICK] How to ban users properly from .htaccess

Started by Ninja ZX-10RR, June 20, 2014, 09:52:04 PM

Previous topic - Next topic

Ninja ZX-10RR

As this started to become a very frequently asked question and I was directly concerned about it I decided to make this guide, hopefully someone of the team will approve it :)

SMF comes with a built-in ban feature that allows admins to use an IP ban to ban users but this function has several problems that cannot be fixed:

  • Each time a user loads a page SMF has to check his IP to see whether he is allowed or not to access that page;
  • SMF has to check all users connected comparing their IPs to the ones in the banned list;
  • Banned users will spam your error log, especially if they are spambots attempting to access the page hundreds of times (100 denied accesses mean 100 errors in the log for ONE spambot);
  • This check will cause your system to terribly slow down;
  • It can't be changed as creating a table and a check is the only way to ban people through SMF software (and any forum software, actually).



How should I ban users?

You should ban users using your .htaccess file that is stored on your server host or either create it. If you don't know how to create it just create a .txt file, put the code in there and when you are done just rename its extension to ".htaccess". ;) Like: example.txt --> rename --> example.htaccess
Open it and add the provided code, better at its end ;)




What should I add to my .htaccess file?

Thanks to Lou69 I can provide these 2 links as reference:
  • http://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#order
  • http://httpd.apache.org/docs/current/mod/mod_access_compat.html#allow
    But as they contain a lot of other stuff I will be explaining the only things that we are looking for.
    Server can be setup to either accept or deny access to any IP not included in the list you are going to make. How? With this:
    order allow,deny or order deny,allow
    Basically the former sets up the server to deny any unlisted entries while the latter sets it up to allow any unlisted entries by default. Be careful because choosing between these two is vital as they order the operations being performed by the server itself. What do I mean? I mean that "allow,deny" will be processing all whitelisted IPs and then banned ones, while "deny,allow" will be doing the opposite. Like I said choosing between one of these is crucial and you have to type everything properly or your site might be unavailable to everybody.




    Let's make some examples, let's say I want to ban this IP 30.31.32.33 and I want any other IP to be allowed on my website, how do I do that? You can do it in 2 ways:
    order allow,deny
    deny from 30.31.32.33
    allow from all
    or either
    order deny,allow
    deny from 30.31.32.33


    As you can see they are slightly different
  • The first one has the "allow from all" in the end as the default state of the server is to deny any unlisted entried
  • The second one doesn't have that string because as the server is processing the banned entries before the allowed ones typing "allow from all" would unban the banned users ( :P )




    How to ban an IP range?

    Let's say I want to ban all users with this IP range 30.31.32.XX where those XX stand for any number. You will have to do this:
    order allow,deny
    deny from 30.31.32.0/24
    allow from all

    or either
    order deny,allow
    deny from 30.31.32.0/24
    I will not be explaining the reasons of that 24 as it's really complicated for new users.

    Another example, I want to ban this IP range: 30.31.XX.XX where those XX stand for any number. You will have to do this:
    order allow,deny
    deny from 30.31.0/16
    allow from all

    or either
    order deny,allow
    deny from 30.31.0/16


    Another example again, I want to ban this IP range: 30.XX.XX.XX where those XX stand for any number. You will have to do this:
    order allow,deny
    deny from 30.0/8
    allow from all

    or either
    order deny,allow
    deny from 30.0/8


    You can get more informations from here: http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#IPv4_CIDR_blocks




    Do you need an exception to unban an IP within a banned IP range?

    Let's say I want this IP range to be banned (30.31.32.XX) but I want this IP (30.31.32.33) to be allowed access. How? (Of course any other user is supposed to be granted access)
    order deny,allow
    deny from 30.31.32.0/24
    allow from 30.31.32.33

    You just can't do it with the "order allow,deny" as it would be processing banned users after allowed ones and it would inevitably banning the IP exception as well.




    Are you running Cloudflare on your forum?

    If you are running Cloudflare on your forum all the above codes will not work properly, you will need to change some things. The "order allow,deny" and "order deny,allow" will work with Cloudflare as well but since Cloudflare is masking all IP addresses as if they were (usually) all 108.162.xxx.xxx then any ban made with the aforementioned method will obviously not work.
    How to ban the proper IP addresses then? You need to add this code:
    SetEnvIf X-FORWARDED-FOR 1.2.3.4 deniedip or SetEnvIf X-FORWARDED-FOR 1.2.3.4 allowedip
    These must be used in ALL ban strings.

    At the end of it you must also add deny from env=deniedip in order to get it to work :) obviously you can change all of them to allow from env=allowedip if you want to restrict access to only a few people.

    CIDR notation (0/8, 0/16, 0/24) will work as long as you add the SetEnvIf X-FORWARDED-FOR code.

    Example 1 (IP 1.2.3.4 is the only banned entry):

    SetEnvIf X-FORWARDED-FOR 1.2.3.4 deniedip
    order allow,deny
    deny from env=deniedip


    Or example 2 (IP 1.2.3.4 is the only IP with granted access):

    SetEnfIf X-FORWARDED-FOR 1.2.3.4 allowedip
    order deny,allow
    deny from all
    allow from env=allowedip


    This part gave me a lot of headaches and in my opinion is the most complicated... I tried to be as clear as possible but if you don't understand it feel free to ask about it :)
    As a reference the Cloudflare part has been adapted from here: http://blog.ergatides.com/2011/09/07/securing-wordpress-dashboard-using-htaccess-behind-cloudflare-or-any-other-cdn/ it's not exactly the same thing but the basic inspiration and reference is there.




    Do not put whitespaces between "order," and "deny" or the opposite, just type "order allow,deny" or "order deny,allow" and no whitespaces next to the comma! Same thing applies to Cloudflare-related strings.




    Related topics and messages:




    Credits to (hopefully I will remind all of you): Arantor, a10, Antechinus, Kindred, br360, emanuele, Shambles and CoreISP, thank you all for your many tweaks and help.

    Special thanks again to Lou69 for the reference links.

    If I forgot anyone please let me know I will apologize and add you immediately.
Quote from: BeastMode topic=525177.msg3720020#msg3720020
It's so powerful that on this post and even in the two PMs you sent me,you still answered my question very quickly and you're apologizing for the delay. You're the #1 support I've probably ever encountered man, so much respect for that. Thank you, and get better soon.

I'll keep this in my siggy for a while just to remind me that someone appreciated what I did while others didn't.

♥ Jess ♥

STOP EDITING MY PROFILE

LiroyvH

Quote
hopefully someone of the team will approve it

There is no moderation in this section.
Anyone is free to write up whatever tricks they wish, if they make sense.
(Although I am wondering whether or not this is the correct board... This is not coding...)

Quote
You should ban users using your .htaccess file that is in your database.
Open it and you will find a bunch of stuff in there regarding things you are not interested in at the moment. Just leave it the way it is, you need to add some code to it, better at its end

.htaccess is in the files, or can be created there. It's not in the database.
It's usually either empty or non-existent by default, "a bunch of stuff" is usually not the case. (Depending, usually, on whether or not there's other software present on the account. Like Wordpress.)

Quote
I will not be explaining the reasons of that 24 as it's really complicated for new users.

Yet it is crucial. CIDR notation can make the difference between banning only a couple of IP's or a complete range.
Don't choose for the user, make them decide for themselves whether they understand or not:
http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#IPv4_CIDR_blocks
It's not that hard. :)

Be careful of banning very large ranges of IP's. In fact, banning IP's on large forums with visitors from multiple countries can actually have adverse affects.


Other than that, the post looks fine although i haven't really looked at the provided examples. ;)
((U + C + I)x(10 − S)) / 20xAx1 / (1 − sin(F / 10))
President/CEO of Simple Machines - Server Manager
Please do not PM for support - anything else is usually OK.

Ninja ZX-10RR

Quote from: BeastMode topic=525177.msg3720020#msg3720020
It's so powerful that on this post and even in the two PMs you sent me,you still answered my question very quickly and you're apologizing for the delay. You're the #1 support I've probably ever encountered man, so much respect for that. Thank you, and get better soon.

I'll keep this in my siggy for a while just to remind me that someone appreciated what I did while others didn't.

♥ Jess ♥

STOP EDITING MY PROFILE

Ninja ZX-10RR

I know that message is very old but... Well if you approve it please move it to tips and tricks :)
And actually last time I checked I wasn't able to post there... Somebody must have changed the permissions for sure and without knowing it I did this the way it was asked in that old message as tips and tricks was usually meant to be some kind of a FAQ section somehow... Bah I don't know. If you think it's good anyhow please move it there as I can't anymore now having already posted it :P
Quote from: BeastMode topic=525177.msg3720020#msg3720020
It's so powerful that on this post and even in the two PMs you sent me,you still answered my question very quickly and you're apologizing for the delay. You're the #1 support I've probably ever encountered man, so much respect for that. Thank you, and get better soon.

I'll keep this in my siggy for a while just to remind me that someone appreciated what I did while others didn't.

♥ Jess ♥

STOP EDITING MY PROFILE

Illori

Quote from: Flavio93Zena on June 20, 2014, 09:52:04 PM
Wait I'm not sure of what I did! How can I try it and see if it works?

You can easily try your code by adding this after all the other commands: defy from XX.XX.XX.XX where those XX stand for the IP you want to test. Refresh the page and if you get a 500 error the IP you just tested is banned. Be quick though and be sure to get rid of this string as this test would deny access to everybody!!! So just defy, refresh, test and then come back and delete it as soon as possible.





do you mean deny or is defy not a typo? also i think you should say that the ip should be yours, not just any random ip.

Ninja ZX-10RR

It is really defy. For testing purposes if you type "defy" it is like "try" in there although the command "try" does not exist :) so I confirm defy command.

And no the IP shouldn't be my own because if the admin wants to check if he has successfully banned a user or not he will have to defy from the (hopefully) banned IP and not his own to test whether that IP is banned or not. If he gets a 500 error then the IP is banned, if he can gain access to the site the IP is not banned. In both cases like I said it's better to do this quick because if the IP address being tested is really banned the whole website will be unavailable for everybody.
Quote from: BeastMode topic=525177.msg3720020#msg3720020
It's so powerful that on this post and even in the two PMs you sent me,you still answered my question very quickly and you're apologizing for the delay. You're the #1 support I've probably ever encountered man, so much respect for that. Thank you, and get better soon.

I'll keep this in my siggy for a while just to remind me that someone appreciated what I did while others didn't.

♥ Jess ♥

STOP EDITING MY PROFILE

byproduct

it is htaccess coding? :)


Quote from: CoreISP on June 20, 2014, 09:58:35 PM
Quote
hopefully someone of the team will approve it

There is no moderation in this section.
Anyone is free to write up whatever tricks they wish, if they make sense.
(Although I am wondering whether or not this is the correct board... This is not coding...)

Quote
You should ban users using your .htaccess file that is in your database.
Open it and you will find a bunch of stuff in there regarding things you are not interested in at the moment. Just leave it the way it is, you need to add some code to it, better at its end

.htaccess is in the files, or can be created there. It's not in the database.
It's usually either empty or non-existent by default, "a bunch of stuff" is usually not the case. (Depending, usually, on whether or not there's other software present on the account. Like Wordpress.)

Quote
I will not be explaining the reasons of that 24 as it's really complicated for new users.

Yet it is crucial. CIDR notation can make the difference between banning only a couple of IP's or a complete range.
Don't choose for the user, make them decide for themselves whether they understand or not:
http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#IPv4_CIDR_blocks
It's not that hard. :)

Be careful of banning very large ranges of IP's. In fact, banning IP's on large forums with visitors from multiple countries can actually have adverse affects.


Other than that, the post looks fine although i haven't really looked at the provided examples. ;)

Kindred

Quote from: Ninja ZX-10RR on June 21, 2014, 06:18:39 AM
It is really defy. For testing purposes if you type "defy" it is like "try" in there although the command "try" does not exist :) so I confirm defy command.

reference?

I can find no evidence on any site that DEFY is actually a valid htaccess command...
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

Ninja ZX-10RR

That's why I updated it this morning - I had a reference that was indeed removed. And whenever I tried it it was giving a 503 error, for sure, but not because it was working, it was doing it because the server was unable to read the string. Now it's correct.
Quote from: BeastMode topic=525177.msg3720020#msg3720020
It's so powerful that on this post and even in the two PMs you sent me,you still answered my question very quickly and you're apologizing for the delay. You're the #1 support I've probably ever encountered man, so much respect for that. Thank you, and get better soon.

I'll keep this in my siggy for a while just to remind me that someone appreciated what I did while others didn't.

♥ Jess ♥

STOP EDITING MY PROFILE

Jade Elizabeth

I never ban by IP, you never know when someone's in a library or using a friends computer. Ban a user, if they come back and behave no problem. If they come back and cause ****** then ban them again and have staff that can do it.

Banning by IP is usually something I do temporarily if I do it at all because you can block legitimate users unintentionally. I once blocked an entire school (8 members) by mistake because someone was being nasty from it.
Once proud Documentation Writer and Help Squad Leader | Check out my new adult coloring career: Color With Jade/Patreon.

Ninja ZX-10RR

It is really useful when it comes to banning spambots and malicious crap trying to register continuously ;) I just look at each one of them and ban them straightaway so that they can't come back and waste more resources 8) it's also good in some cases if an annoying user creates nultiple accounts and he doesn't know how to use browsers like Tor or similar services.
Quote from: BeastMode topic=525177.msg3720020#msg3720020
It's so powerful that on this post and even in the two PMs you sent me,you still answered my question very quickly and you're apologizing for the delay. You're the #1 support I've probably ever encountered man, so much respect for that. Thank you, and get better soon.

I'll keep this in my siggy for a while just to remind me that someone appreciated what I did while others didn't.

♥ Jess ♥

STOP EDITING MY PROFILE

Jade Elizabeth

Waste of time. Install Stop Spammer and HTTPBL. They stop bots and spammers from registering without any additional effort from yourself.

http://custom.simplemachines.org/mods/?mod=1547
http://custom.simplemachines.org/mods/index.php?mod=2155

For more persistent bots add this to your httaccess:

# BLOCK USER AGENTS
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} AhrefsBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} spbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DigExt [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Sogou [NC,OR]
RewriteCond %{HTTP_USER_AGENT} MJ12 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} majestic12 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} 80legs [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SISTRIX [NC,OR]
RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Semrush [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Ezooms [NC,OR]
RewriteCond %{HTTP_USER_AGENT} CCBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} TalkTalk [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Ahrefs [NC]
RewriteRule !^robots\.txt$ - [F]

# BLOCK BLANK USER AGENTS
RewriteCond %{HTTP_USER_AGENT} ^-?$

Once proud Documentation Writer and Help Squad Leader | Check out my new adult coloring career: Color With Jade/Patreon.

Ninja ZX-10RR

Nope about the mods, I have installed almost any anti-spam existing on the mod site.
Also I want to ban them from the .htaccess, but anyway the code you have provided is useful for that, thanks :) I will try it when I will be home on the 25 or either 26 if I won't be online the 25.
Quote from: BeastMode topic=525177.msg3720020#msg3720020
It's so powerful that on this post and even in the two PMs you sent me,you still answered my question very quickly and you're apologizing for the delay. You're the #1 support I've probably ever encountered man, so much respect for that. Thank you, and get better soon.

I'll keep this in my siggy for a while just to remind me that someone appreciated what I did while others didn't.

♥ Jess ♥

STOP EDITING MY PROFILE

Ninja ZX-10RR

Updated:

  • Improved topic layout;
  • Corrected a typo at the end of the post;
  • Added all the Cloudflare part;
  • Edited the credits;
  • Explained how to create an .htaccess file;
  • Removed a wrong piece of the post that was due to another page being completely incorrect (defy).

It should be "perfect" now. :D
Quote from: BeastMode topic=525177.msg3720020#msg3720020
It's so powerful that on this post and even in the two PMs you sent me,you still answered my question very quickly and you're apologizing for the delay. You're the #1 support I've probably ever encountered man, so much respect for that. Thank you, and get better soon.

I'll keep this in my siggy for a while just to remind me that someone appreciated what I did while others didn't.

♥ Jess ♥

STOP EDITING MY PROFILE

DavidGreen

Quote from: Jade Elizabeth on August 21, 2014, 08:55:01 PM
I never ban by IP, you never know when someone's in a library or using a friends computer. Ban a user, if they come back and behave no problem. If they come back and cause ****** then ban them again and have staff that can do it.

Banning by IP is usually something I do temporarily if I do it at all because you can block legitimate users unintentionally. I once blocked an entire school (8 members) by mistake because someone was being nasty from it.

That's a good point. Some people have the same ip so just banning an ip can end up getting rid of legit users. And lots of spammers are getting through on my forum even though I ask tough questions on my registration form so I guess they are real people manually signing up.

Right now I have no choice but manually approve messages before they get posted but I feel this stops the flow of chat as sometimes it can take me hours before I approve a comment so the legit users are suffering.

Thanks for the tutorial Ninja ZX-10RR I'll give this a go.

Advertisement: