News:

SMF 2.1.4 has been released! Take it for a spin! Read more.

Main Menu

Banning By htaccess

Started by br360, August 14, 2013, 02:08:16 AM

Previous topic - Next topic

br360

When I first started my site a while back, I had a ton of spammers, and not knowing any better, I simply just banned them through my admin panel. Well I have been reading a few posts here and noticed that this slows down performance significantly, and should have been done through my htaccess.

Well 2+ years later, and I have like 200+ pages of ip bans in my banlist now.  :o

I want to move them to my htaccess, but of course want to make sure I am doing it right, as well as finding out if putting in that many ip's will cause any serious issues with my htaccess file.

This is what the file looks like now

rewriteengine on
rewritecond %{HTTPS} off
rewritecond %{HTTP_HOST} ^www.mysite.com$ [OR]
rewritecond %{HTTP_HOST} ^mysite.com$
rewriterule ^$ "https\:\/\/mysite\.com\/" [R=301,L] #5(other numbers and letters)


Is this what I should be doing? I gave an example of 4 ip addresses, but if I have literally hundreds of ip bans, should I just list them all in a row like I did with the 4 below?

[code]rewriteengine on
rewritecond %{HTTPS} off
rewritecond %{HTTP_HOST} ^www.mysite.com$ [OR]
rewritecond %{HTTP_HOST} ^mysite.com$
rewriterule ^$ "https\:\/\/mysite\.com\/" [R=301,L] #5(other numbers and letters)

order allow,deny
deny from 201.68.101.5
deny from 110.38.202.5
deny from 110.55.222.3
deny from 110.48.104.9
allow from all


Or now that I have much better spam prevention, would it be easier just to start deleting the ip's from my ban list in my admin panel; as they now aren't able to get back onto the board or even register for that matter?




Chalky

I'd just delete them all and start again adding any new pests to .htaccess as they come along. If your spam prevention mods are adequate then, as you say, the spammers won't even get close anyway. In fact most of your bans have probably never been back since you banned them.  No, not worth the effort, just remove them all and start again with .htaccess (or your cPanel's deny manager which is easier).

br360

Thanks. That is what I was hoping for too as it's a lot easier to not screw up. ;)

Dumb question; but I'm guessing deleting my 50 pages of ban triggers is ok to do as well?


Arantor

Sure it is. You'll find your forum will be faster too!

In fact if you look, don't be surprised if the bans you've set up never actually get hit in the first place.

Illori

if you do have better spam prevention then the spammers should not even get to your forum. so banning them is not an issue.

http://wiki.simplemachines.org/smf/Spam_-_my_forum_is_flooded_with_spam,_what_can_I_do

br360

Appreciate the help. I'll mark this as solved.

ARG01

Quote...or your cPanel's deny manager which is easier

This is what I do. This will also keep pests away from any sub-directories as well such as siteurl/forum, siteurl/gallery, etc.
No, I will not offer free downloads to Premium DzinerStuido themes. Please stop asking.

MrPhil

"An ounce of prevention is worth a pound of cure" -- using effective anti-spam settings and mods will help prevent a lot of spammers from getting a toehold on your system anyway. Once they're in, you can immediately put them in .htaccess. Maybe they'll figure out that they've been banned by the fact that they are rudely bounced out of your site. An alternative, if you want them to know they've been banned, is to use SMF's ban facility, and after 3 or 6 months delete the ban and the member, and add the IP address to /.htaccess's DENY list. This may have the advantage of being quicker to add each ban, and then be able to do a whole bunch at once in editing your /.htaccess, but your mileage may vary. You might also be able to look at how often they tried to get in (the ban was exercised), and decide that they went away after a short time and it's not worth adding them to the DENY list. There are lots of things you can do. There is presumably a performance hit in letting your DENY list get huge, although not as much as using SMF's ban list.

br360

This is odd-  I have been clearing my ban list one page at a time. There are 20 entries per page, and when I do clear a page of 20 at a time, about half of them ( although sometimes less than half, sometimes more than half) reappear as new members waiting for approval. I of course reject these members, and when I do, my membership stats go up on forum stats on the board index. (ex: I go from say 4,522 members to 4,532 each time I clear a page in my ban list)

Oh and by the way, the membership increase is only happening on my index above the who is online list. When I go into my "more stats" link and see how many members have registered today in my Forum history, the membership numbers do not increase there.

Anyone have any ideas why my membership stats are going up?

MrPhil

Are you deleting banned members when you delete a ban? If you don't, they'll just be right back in your face as soon as they realize the 'cuffs are off. I'm not sure if banned members count in the membership numbers or just unbanned ones. Depending on what criteria you've been banning by, you could be opening the gates for others sharing the same IP address, host, email domain, or whatever. I suppose it even be the spammer again, signing up again from the same address, host, etc., and the ban just kept them invisible for a while. Just purge them if the show up again as a registrant.

I wish there was a cleaner way to permanently ban these jokers, without so much manual labor. Something to either ban and remove their posts, or immediately erase them (and their posts) and update the .htaccess file.

Arantor

In the case of spammers, prevention is infinitely better than clean-up. The mass adding of IP addresses is not actually as smart as everyone seems to believe.

Let me explain. There is a forum, running 1.1.x, that I host but I largely leave the admins to it. Now, I checked the ban list recently. All 2,470 pages of it. Yeah, someone's been busy.

Anyway, virtually every IP trigger was never hit. This means that once banned, the IPs were not reused. The final count of IP bans that were never subsequently hit was 94%. That's a LOT of performance headache right there.


In other news, when you ban a member, they don't show on the memberlist any more.

br360

@Mr.Phil- the bans that filled up my banlist were mostly done over the last 2 years and before I actually explored different prevention options. Back then, a spammer would try to register and I would automatically ban him by ip/email/handle. Now with the spam precautions I have put in place, I get maybe one or two spammers a day; as opposed to over 100 a day from just a few months ago.

I have also since deleted about 40 pages of bans from my banlist, and so far not one of them have come back.

Quote from: ArantorIn other news, when you ban a member, they don't show on the memberlist any more.

That I understand, but when they get unbanned, and then pop up on my new members waiting for approval, I reject them. Once rejected, they shouldn't be adding to my members list.

I know that everytime a member registers it gets counted in the forum history- even if they get rejected, but I guess I am not understanding why they are adding to my members in the board index as well. 

Arantor

Because the board index count is the total count of registrations. It's weird, it's stupid but that's how SMF has always operated.

br360

Just wanted to give a follow up for anyone else that may see this in the future- I started out with over 200 pages on my ban list. I was deleting a page at a time (20 members), and it was literally freezing the entire board for about a minute.

As the pages got smaller, the freeze time was less and less- at 150 pages there was about 30 seconds of freeze time when unbanning a page of members, 100 pages there was about 17 seconds of lag, 50 pages, there was about 5 seconds (lol- yes I was timing it)

Well now I have about 4 pages, and not only is there no lag at all if I ban or unban a member, but the board is a lot faster between pages as well.

So moral of the story- do not mass ban spammers as it does take a huge toll on the board's speed. ;)

Thanks again guys for the help. Topic solved.

Chalky

woohoo!!  Glad to hear your forum is feeling the benefits  :D  Thanks for reporting back  :)

byproduct


you opened a whole new can of worms for yourself.
your going to find that htaccess is a powerful DOUBLE EDGED sword.
it rocks, but have ya ever been hit in the head with a rock?


learn about reverse IP whois and CIDR's

whois every visitor, you'll LIKELY find you got a lot of bots visiting your site.
AND MOST OF THOSE ARE *WORSE THAN* USELESS TO YOU

VERY SMALL EXERT from my own htacess
deny from 109.0.0.0/8
deny from 110.0.0.0/8
deny from 111.0.0.0/8
deny from 112.0.0.0/8
deny from 113.0.0.0/8
deny from 114.0.0.0/8
deny from 115.0.0.0/8
deny from 116.0.0.0/8
deny from 117.0.0.0/8
deny from 118.0.0.0/8
deny from 119.0.0.0/8
deny from 120.0.0.0/8
deny from 121.0.0.0/8
deny from 122.0.0.0/8
deny from 123.0.0.0/8
deny from 124.0.0.0/8
deny from 125.0.0.0/8
deny from 126.0.0.0/8

Atrus

I too, made the mistake of banning spam bots through the SMF ban list. We had 12 pages of bans. I just got some time, so I started moving bans from the ban list to .htaccess.

We have a small forum dedicated to western North Carolina. We constantly get bombed with "requests for membership" from China, Russia, Ukraine, and many other countries around the globe." I am currently going through our ban list, and removing one at a time, and adding the ip/iprange to our .htaccess file. There was not one on our server, so I created one. I am finding that many countries have several ip ranges.  I am wondering if there is a way that I can ban an entire country with just a single entry? No one in any foreign country would have any interest in our local forum. I learned that I can ban ranges of ip addresses, and servers, but I haven't seen anything about banning hosts? (*.ru, or *.cn)

Since there was not a .htaccess file on our server, I am a little confused as to exactly where to put it. We host 3 different websites on our one server account, with only one having SMF forums. There is a front-end website, and you reach the forums through the website (of course, the SMF forums can be accessed directly, by copying the url once you reach them, and returning directly to that url). I have uploaded the .htaccess file to the root directory of the SMF forums, as that is the only thing I want to protect from spam bots. Is this what I need to be doing?

The more I learn about SMF the more I appreciate it. 

Thanks, and Merry Christmas to all!


a10

#17
Quoteban an entire country with just a single entry?
None that I've heard about.

For a period I used ranges from websites like https://www.countryipblocks.net/ and tried .htacessing half the world population. Ended up with a .htaccess with over 17.000 lines of ip ranges, worked, not a cn, ru, ua etc ip in sight, but also got into server problems > loss of speed + random wrong 403's, members (and myself) could sometimes not get trough even with the ip definitely not being in the list. Gave up, and went back to using .htaccess for just the most obvious bandwidth thieves + a collection of the worst spam ip's.

I put the .htaccess in the root folder, example mydomain.com, and it will cover mydomain.com/forum and  forum.mydomain.com

Regarding spam registrations (looking at logs, up to 5000 attempts\week), zero problem, taken care of by using a few good registration questions + checking new members with a quick look-up on stopforumspam before accepting. And mods are available for those who need more control.

With a locally oriented forum, you should have no problem choosing some questions where any & every legit member should know what to reply, and quite impossible for the bots\spammers to figure out.
2.0.19, php 8.0.23, MariaDB 10.5.15. Mods: Contact Page, Like Posts, Responsive Curve, Search Focus Dropdown, Add Join Date to Post.

Atrus


One of the things I liked in the ban list was the ability to ban hosts. I could ban the entire country of China with a ban host name of *.cn  Russia with *.ru and so on. Looking at the logs, many of those triggers got thousands of hits. I wish there were a way to do the same in the .htaccess file. I just finished emptying my ban list, and putting most of those banned ip's in the .htaccess file. I have about 150 or so lines, and have most of the world banned. I went through our member list and made sure all registered members' ip addresses were not banned. So far, no problems. We'll see what happens in the next few days.

We have a website on the front end, and one would normally access the forums through the website. Since there is no input anywhere on the website, I put the htaccess file in the root directory of the forums. It should have no effect on the website, only the forums.

Thanks for your reply.

Merry Christmas,

Jim  -  Atrus


byproduct


1st off, htaccess can be dangerous for you.
learn how to protect that file before you start doing thinsg with it
then be careful or you'll block yourself.

Quote from: Atrus on December 16, 2013, 10:36:53 PM
I put the htaccess file in the root directory of the forums. It should have no effect on the website, only the forums.

Couple of points for you....
you can put a htaccess file in ANY directory... i will effect that directory and every file & directory under it.
NOW, server has a setting that can adjust HOW DEEP that effect goes, so that is adjustable, but best not messed with unless your really up on what your doing.

Quote from: Atrus on December 16, 2013, 10:36:53 PM
I could ban the entire country of China with a ban host name of *.cn  Russia with *.ru and so on. Looking at the logs, many of those triggers got thousands of hits. I wish there were a way to do the same in the .htaccess file.
IT CAN BE DONE...
but it's tricky and hazardous to your server/site traffic, and sometimes operation... depending where you & your server is located and what/how you operate.

I block ever country except US, off and on, canada

I can do it in server CSF or htaccess or a combination of both.
1.0.0.0/8 THRU 256.0.0.0/8 covers every possible IP.
some of those assign to 1 country or another, or even multiple countries.

MOST OF THOSE aren't u.s. based.

trick is knowing which assign to where.
WEB SEARCH "IP by country"
takes some digging to find a list by IP "CIDR" per country, FOR FREE.
but it's out there

USE CIDR FORMAT or you'll bog yaself down.
even then, it's larger than most NON DEDICATED servers will like... hosters will squeal.


SERVER CSF FORMAT IS SIMPLY
1.0.0.0/8
2.0.0.0/
so on...

HTACCESS IS
deny from 1.0.0.0/8
deny from 2.0.0.0/8
deny from 5.0.0.0/8
so on...

NOTHER HELPFUL HTACCESS BIT IS...
at very top of hatccess file you can add....
#
#SetEnvIfNoCase Referer  (cloud|host) getout
#
SetEnvIf Request_URI "^(/403.*\.htm|/robots\.txt)$" allowsome
<Files *>
order deny,allow
deny from env=getout
allow from env=allowsome
</Files>

WHAT THIS DOES IS... BLOCK ANY VISITOR that has cloud OR host anywhere in it's visitor info... might be where it came form, it's user aganet, AND DEEPENDING ON SERVER SETTINGS, might be who host their connection..
so it assigns them a liable, "getout"
denyfrom env_getout blocks anyone with that lable.
allow from env_ allowsome lets them view your 403 file and your robots txt file as specified in that send line of instruction.

BEAUTIFUL PART ABOUT THAT CODE IS. YOU CAN ADD "KEYWORDS" TO IT AT WILL...
simply add, IN FRONT OF ")" |NEW KEYWORD
SO YOU HAVE....
(cloud|host|new keyword) getout

BTW, SetEnvIfNoCase THE "NoCase" part means what ever entries you put in are not case sensitive, thus  it will catch "BOT" and "bot" "googleBot" "bingbot" "china-bot" equally.... anything with "bot" ANYWHERE in it

so anyway try adding that to your htaccess and watch you error log & visitor log for a bit... you'll start seeing people getting errors, check their ip in ya visitor log and you should see they got the error because they got a 403 page because, one those keywords was either i their referrer or user agent tags.

IMPORTANT: DO NOT USE A KEYWORD THAT IS PART OF YOUR SITE OR URL STRUCTURE...


hope that helps a bit.

LEARN ABOUT HTACCESS... IT CAN HELP YOU QUITE A BIT
can screw you up too!


Advertisement: