Forum getting swarmed by 500+ guests?

[Rhindeer]

Currently a bunch of IPs from Huawei are going to town. xD Jeez, it's like you take care of one issue and a million others crop up! We got down to 100 guests yesterday after using the list, now it's back into the 500s. Ahhh!

[shawnb61]

Yep.  I analyze the logs about once a week...

[lather]

Thanks for this. I edited my htaccess file and guests dropped from 490 to 40.

[HITG]

Just for everyone interested a premium dns and crawlers delay code did the job. Also DDoS protection might be good for this but so far they are slowly coming in.

[Rhindeer]

It ramped up to the 1700s now. Lots of bots from Vietnam? I'm trying to figure out how to use the tools to squash these. Can't get Cloudflare, but may look into DNS? But dang they got aggressive.

[a10]

9337 guests  approx 90% South America + Vietnam and a few more asians. Individual ip's, no ranges to be spotted. Looks like hacked personal \ business computers \ devices. And among all that, ranges from amazon, huawei, alibaba etc.

No impact on general forum behaviour but have set it to login only for a while. Expecting things will calm down in a few days.

It's the new reality, ai \ scrapers. In earlier times, the plague was bot's registration attempts, they have basically disappeared on my forum.

[sudoku]

If nothing else, it makes your forum appear "busy" and lively. 
My forum was near 5000 just recently. It used to only have 100-150 guests at any given time - for years...

[Rhindeer]

9337 guests  approx 90% South America + Vietnam and a few more asians. Individual ip's, no ranges to be spotted. Looks like hacked personal \ business computers \ devices. And among all that, ranges from amazon, huawei, alibaba etc.

No impact on general forum behaviour but have set it to login only for a while. Expecting things will calm down in a few days.

It's the new reality, ai \ scrapers. In earlier times, the plague was bot's registration attempts, they have basically disappeared on my forum.

I set mine to login only and the numbers keep increasing still. D8 Maybe I did it wrong, what was your process?

They keep timing out my forum.

[a10]

I set mine to login only and the numbers keep increasing still. D8 Maybe I did it wrong, what was your process?

They keep timing out my forum.

Bots will still be there and trying but won't access anything except the welcome page. Stopping guests should\will\may hopefully restrict extreme use of bandwidth and server resources. Am on a shared server so host would maybe react, but no problems yet.

[shawnb61]

The problem is that if it hits your forum at all there is still a lot of work to do: connect to db & load settings; look for login cookie/session, maybe create a session; load the theme, maybe update stats, etc.

Under most circumstances, not a problem.  When you're flooded with activity, it's a problem.

That's why you have to filter them out BEFORE they even get to load a page, via .htaccess.

I haven't seen these guys (yet), so they're not added to my .htaccess starter pack:
https://github.com/sbulen/SMF-bot-hygiene

I would see which IPs/CIDRs are hitting you the most & add to .htaccess.

[a10]

If of interest, attached a sample from some minutes of activity this evening. This is very few of the different ip's in use.

[shawnb61]

If of interest, attached a sample from some minutes of activity this evening. This is very few of the different ip's in use.

I've looked up about a dozen of those, & it's a mix of:
 - AWS AS16509, AS14618
 - Indonesia AS138064, AS63859
 - Indonesia - some folks I couldn't find the ASN
 - Vietnam - AS45899

Of these, I think the Vietnam ASN covers the most of the IPs.

So...  One way to get a bunch of the CIDRs is to go to this site:
https://hackertarget.com/as-ip-lookup/

And enter the ASN: AS45899

The list you'll get doing so covers a lot of the IPs in the list provided.

The question is whether you have users there.  When I have one or two IPs, I just simply use the ACP admin user search & look up users by the first couple of nodes of an offending IP, e.g., "113.172" and see if I get any hits.

For a much bigger task like this, I actually download a copy of my smf_members into Excel & do a VLookup of member IPs against these ranges. 

If I find no matches, it's a no brainer.  Block 'em all.  Most of these attacks seem to come from commercial datacenters, not end-user ISPs.  Just block 'em.   

If I find matches, so blocking them may cut off existing users, it's a much harder decision...

(I'm on the road so can't do so or update my shared version of .htaccess for another week or so...)

[shawnb61]

I don't have all my utilities at my disposal (out of town), but if you're basically down, it might be worth trying this.  It has not been cleaned & de-duped & checked against users, etc.  This ASN appears to cover a big hunk of Vietnam. 

AS45899:

Code Select Expand

Deny from 103.10.44.0/22
Deny from 103.107.180.0/22
Deny from 103.120.242.0/23
Deny from 103.131.68.0/22
Deny from 103.148.56.0/23
Deny from 103.15.90.0/23
Deny from 103.154.244.0/23
Deny from 103.171.92.0/24
Deny from 103.197.184.0/23
Deny from 103.237.96.0/22
Deny from 103.238.214.0/24
Deny from 103.238.215.0/24
Deny from 103.255.239.0/24
Deny from 103.42.56.0/24
Deny from 103.42.57.0/24
Deny from 103.42.58.0/24
Deny from 103.42.59.0/24
Deny from 103.48.191.0/24
Deny from 103.53.168.0/24
Deny from 103.53.169.0/24
Deny from 103.56.156.0/22
Deny from 103.68.252.0/22
Deny from 103.69.192.0/23
Deny from 103.89.92.0/22
Deny from 103.9.76.0/22
Deny from 103.9.76.0/24
Deny from 103.9.77.0/24
Deny from 103.9.78.0/24
Deny from 103.9.79.0/24
Deny from 113.160.0.0/16
Deny from 113.161.0.0/16
Deny from 113.162.0.0/16
Deny from 113.163.0.0/16
Deny from 113.164.0.0/16
Deny from 113.165.0.0/16
Deny from 113.166.0.0/16
Deny from 113.167.0.0/16
Deny from 113.168.0.0/16
Deny from 113.169.0.0/16
Deny from 113.170.0.0/16
Deny from 113.171.0.0/16
Deny from 113.172.0.0/16
Deny from 113.173.0.0/16
Deny from 113.174.0.0/16
Deny from 113.175.0.0/16
Deny from 113.176.0.0/16
Deny from 113.177.0.0/16
Deny from 113.178.0.0/16
Deny from 113.179.0.0/16
Deny from 113.180.0.0/16
Deny from 113.181.0.0/16
Deny from 113.182.0.0/16
Deny from 113.183.0.0/16
Deny from 113.184.0.0/16
Deny from 113.185.0.0/16
Deny from 113.186.0.0/16
Deny from 113.187.0.0/16
Deny from 113.188.0.0/16
Deny from 113.189.0.0/16
Deny from 113.190.0.0/16
Deny from 113.191.0.0/16
Deny from 123.16.0.0/16
Deny from 123.17.0.0/16
Deny from 123.18.0.0/16
Deny from 123.19.0.0/16
Deny from 123.20.0.0/16
Deny from 123.21.0.0/16
Deny from 123.22.0.0/16
Deny from 123.23.0.0/16
Deny from 123.24.0.0/16
Deny from 123.25.0.0/16
Deny from 123.26.0.0/16
Deny from 123.27.0.0/16
Deny from 123.28.0.0/16
Deny from 123.29.0.0/16
Deny from 123.30.0.0/18
Deny from 123.30.0.0/20
Deny from 123.30.0.0/21
Deny from 123.30.0.0/23
Deny from 123.30.10.0/23
Deny from 123.30.10.0/24
Deny from 123.30.100.0/23
Deny from 123.30.102.0/23
Deny from 123.30.106.0/23
Deny from 123.30.106.0/24
Deny from 123.30.109.0/24
Deny from 123.30.112.0/20
Deny from 123.30.112.0/21
Deny from 123.30.112.0/23
Deny from 123.30.114.0/23
Deny from 123.30.116.0/23
Deny from 123.30.118.0/23
Deny from 123.30.12.0/23
Deny from 123.30.120.0/22
Deny from 123.30.120.0/23
Deny from 123.30.124.0/23
Deny from 123.30.128.0/18
Deny from 123.30.128.0/20
Deny from 123.30.130.0/23
Deny from 123.30.14.0/23
Deny from 123.30.141.0/24
Deny from 123.30.144.0/20
Deny from 123.30.144.0/22
Deny from 123.30.144.0/23
Deny from 123.30.144.0/24
Deny from 123.30.145.0/24
Deny from 123.30.146.0/23
Deny from 123.30.146.0/24
Deny from 123.30.147.0/24
Deny from 123.30.148.0/23
Deny from 123.30.148.0/24
Deny from 123.30.151.0/24
Deny from 123.30.152.0/23
Deny from 123.30.152.0/24
Deny from 123.30.153.0/24
Deny from 123.30.16.0/20
Deny from 123.30.16.0/21
Deny from 123.30.16.0/23
Deny from 123.30.160.0/20
Deny from 123.30.162.0/24
Deny from 123.30.166.0/23
Deny from 123.30.168.0/21
Deny from 123.30.168.0/23
Deny from 123.30.168.0/24
Deny from 123.30.169.0/24
Deny from 123.30.170.0/23
Deny from 123.30.170.0/24
Deny from 123.30.171.0/24
Deny from 123.30.172.0/23
Deny from 123.30.172.0/24
Deny from 123.30.173.0/24
Deny from 123.30.174.0/23
Deny from 123.30.174.0/24
Deny from 123.30.175.0/24
Deny from 123.30.176.0/20
Deny from 123.30.176.0/21
Deny from 123.30.176.0/23
Deny from 123.30.176.0/24
Deny from 123.30.177.0/24
Deny from 123.30.178.0/23
Deny from 123.30.178.0/24
Deny from 123.30.179.0/24
Deny from 123.30.18.0/23
Deny from 123.30.180.0/23
Deny from 123.30.180.0/24
Deny from 123.30.181.0/24
Deny from 123.30.182.0/23
Deny from 123.30.182.0/24
Deny from 123.30.183.0/24
Deny from 123.30.184.0/21
Deny from 123.30.184.0/23
Deny from 123.30.184.0/24
Deny from 123.30.185.0/24
Deny from 123.30.186.0/23
Deny from 123.30.186.0/24
Deny from 123.30.187.0/24
Deny from 123.30.188.0/23
Deny from 123.30.188.0/24
Deny from 123.30.189.0/24
Deny from 123.30.190.0/23
Deny from 123.30.190.0/24
Deny from 123.30.191.0/24
Deny from 123.30.192.0/18
Deny from 123.30.192.0/20
Deny from 123.30.192.0/21
Deny from 123.30.192.0/23
Deny from 123.30.2.0/23
Deny from 123.30.20.0/23
Deny from 123.30.202.0/23
Deny from 123.30.204.0/22
Deny from 123.30.208.0/20
Deny from 123.30.208.0/21
Deny from 123.30.208.0/23
Deny from 123.30.208.0/24
Deny from 123.30.209.0/24
Deny from 123.30.210.0/23
Deny from 123.30.210.0/24
Deny from 123.30.211.0/24
Deny from 123.30.212.0/23
Deny from 123.30.212.0/24
Deny from 123.30.214.0/23
Deny from 123.30.214.0/24
Deny from 123.30.215.0/24
Deny from 123.30.216.0/21
Deny from 123.30.217.0/24
Deny from 123.30.22.0/23
Deny from 123.30.223.0/24
Deny from 123.30.224.0/20
Deny from 123.30.224.0/23
Deny from 123.30.226.0/23
Deny from 123.30.228.0/23
Deny from 123.30.232.0/21
Deny from 123.30.232.0/23
Deny from 123.30.232.0/24
Deny from 123.30.233.0/24
Deny from 123.30.234.0/23
Deny from 123.30.234.0/24
Deny from 123.30.235.0/24
Deny from 123.30.236.0/23
Deny from 123.30.236.0/24
Deny from 123.30.237.0/24
Deny from 123.30.238.0/23
Deny from 123.30.238.0/24
Deny from 123.30.239.0/24
Deny from 123.30.24.0/21
Deny from 123.30.24.0/23
Deny from 123.30.240.0/20
Deny from 123.30.240.0/23
Deny from 123.30.240.0/24
Deny from 123.30.241.0/24
Deny from 123.30.245.0/24
Deny from 123.30.246.0/23
Deny from 123.30.246.0/24
Deny from 123.30.247.0/24
Deny from 123.30.248.0/23
Deny from 123.30.249.0/24
Deny from 123.30.251.0/24
Deny from 123.30.26.0/23
Deny from 123.30.28.0/23
Deny from 123.30.29.0/24
Deny from 123.30.30.0/23
Deny from 123.30.32.0/20
Deny from 123.30.32.0/21
Deny from 123.30.32.0/23
Deny from 123.30.34.0/23
Deny from 123.30.34.0/24
Deny from 123.30.36.0/23
Deny from 123.30.37.0/24
Deny from 123.30.38.0/23
Deny from 123.30.4.0/23
Deny from 123.30.40.0/21
Deny from 123.30.40.0/23
Deny from 123.30.40.0/24
Deny from 123.30.41.0/24
Deny from 123.30.42.0/23
Deny from 123.30.42.0/24
Deny from 123.30.48.0/20
Deny from 123.30.48.0/21
Deny from 123.30.48.0/23
Deny from 123.30.48.0/24
Deny from 123.30.49.0/24
Deny from 123.30.50.0/23
Deny from 123.30.50.0/24
Deny from 123.30.51.0/24
Deny from 123.30.52.0/23
Deny from 123.30.53.0/24
Deny from 123.30.54.0/23
Deny from 123.30.56.0/21
Deny from 123.30.56.0/23
Deny from 123.30.57.0/24
Deny from 123.30.58.0/23
Deny from 123.30.58.0/24
Deny from 123.30.59.0/24
Deny from 123.30.6.0/23
Deny from 123.30.60.0/23
Deny from 123.30.60.0/24
Deny from 123.30.61.0/24
Deny from 123.30.62.0/23
Deny from 123.30.64.0/18
Deny from 123.30.64.0/20
Deny from 123.30.64.0/23
Deny from 123.30.66.0/23
Deny from 123.30.66.0/24
Deny from 123.30.68.0/23
Deny from 123.30.74.0/23
Deny from 123.30.74.0/24
Deny from 123.30.75.0/24
Deny from 123.30.76.0/23
Deny from 123.30.78.0/23
Deny from 123.30.8.0/21
Deny from 123.30.8.0/23
Deny from 123.30.8.0/24
Deny from 123.30.80.0/20
Deny from 123.30.80.0/21
Deny from 123.30.82.0/23
Deny from 123.30.86.0/23
Deny from 123.30.88.0/23
Deny from 123.30.9.0/24
Deny from 123.30.90.0/23
Deny from 123.30.92.0/23
Deny from 123.30.96.0/20
Deny from 123.30.96.0/21
Deny from 123.30.96.0/23
Deny from 123.30.96.0/24
Deny from 123.30.97.0/24
Deny from 123.30.98.0/23
Deny from 123.31.0.0/18
Deny from 123.31.0.0/21
Deny from 123.31.110.0/24
Deny from 123.31.111.0/24
Deny from 123.31.112.0/20
Deny from 123.31.113.0/24
Deny from 123.31.128.0/18
Deny from 123.31.128.0/20
Deny from 123.31.14.0/23
Deny from 123.31.144.0/20
Deny from 123.31.15.0/24
Deny from 123.31.156.0/24
Deny from 123.31.16.0/21
Deny from 123.31.16.0/24
Deny from 123.31.160.0/20
Deny from 123.31.17.0/24
Deny from 123.31.176.0/20
Deny from 123.31.18.0/24
Deny from 123.31.184.0/23
Deny from 123.31.186.0/23
Deny from 123.31.19.0/24
Deny from 123.31.192.0/18
Deny from 123.31.192.0/20
Deny from 123.31.20.0/24
Deny from 123.31.208.0/20
Deny from 123.31.21.0/24
Deny from 123.31.22.0/24
Deny from 123.31.224.0/20
Deny from 123.31.23.0/24
Deny from 123.31.240.0/20
Deny from 123.31.25.0/24
Deny from 123.31.26.0/24
Deny from 123.31.32.0/19
Deny from 123.31.48.0/20
Deny from 123.31.64.0/18
Deny from 123.31.64.0/20
Deny from 123.31.64.0/23
Deny from 123.31.66.0/23
Deny from 123.31.8.0/24
Deny from 123.31.80.0/20
Deny from 123.31.9.0/24
Deny from 123.31.96.0/20
Deny from 123.31.96.0/23
Deny from 123.31.98.0/23
Deny from 14.160.0.0/16
Deny from 14.161.0.0/16
Deny from 14.162.0.0/16
Deny from 14.163.0.0/16
Deny from 14.164.0.0/16
Deny from 14.165.0.0/16
Deny from 14.166.0.0/16
Deny from 14.167.0.0/16
Deny from 14.168.0.0/16
Deny from 14.169.0.0/16
Deny from 14.170.0.0/16
Deny from 14.171.0.0/16
Deny from 14.172.0.0/16
Deny from 14.173.0.0/16
Deny from 14.174.0.0/16
Deny from 14.175.0.0/16
Deny from 14.176.0.0/16
Deny from 14.177.0.0/16
Deny from 14.178.0.0/16
Deny from 14.179.0.0/16
Deny from 14.180.0.0/16
Deny from 14.181.0.0/16
Deny from 14.182.0.0/16
Deny from 14.183.0.0/16
Deny from 14.184.0.0/16
Deny from 14.185.0.0/16
Deny from 14.186.0.0/16
Deny from 14.187.0.0/16
Deny from 14.188.0.0/16
Deny from 14.189.0.0/16
Deny from 14.190.0.0/16
Deny from 14.191.0.0/16
Deny from 14.224.0.0/16
Deny from 14.226.0.0/16
Deny from 14.227.0.0/16
Deny from 14.228.0.0/16
Deny from 14.229.0.0/16
Deny from 14.230.0.0/16
Deny from 14.231.0.0/16
Deny from 14.232.0.0/16
Deny from 14.233.0.0/16
Deny from 14.234.0.0/16
Deny from 14.235.0.0/16
Deny from 14.236.0.0/16
Deny from 14.237.0.0/16
Deny from 14.238.0.0/16
Deny from 14.239.0.0/16
Deny from 14.240.0.0/16
Deny from 14.241.0.0/16
Deny from 14.242.0.0/16
Deny from 14.243.0.0/16
Deny from 14.244.0.0/16
Deny from 14.245.0.0/16
Deny from 14.246.0.0/16
Deny from 14.247.0.0/16
Deny from 14.248.0.0/16
Deny from 14.249.0.0/16
Deny from 14.250.0.0/16
Deny from 14.251.0.0/16
Deny from 14.252.0.0/16
Deny from 14.253.0.0/16
Deny from 14.254.0.0/16
Deny from 14.255.0.0/16
Deny from 146.196.64.0/24
Deny from 146.196.65.0/24
Deny from 146.196.66.0/24
Deny from 146.196.67.0/24
Deny from 157.119.246.0/23
Deny from 157.119.246.0/24
Deny from 157.119.247.0/24
Deny from 157.66.13.0/24
Deny from 160.19.78.0/23
Deny from 160.191.52.0/23
Deny from 160.22.16.0/23
Deny from 160.22.2.0/24
Deny from 160.22.3.0/24
Deny from 160.25.74.0/23
Deny from 160.250.216.0/23
Deny from 160.30.54.0/23
Deny from 161.248.30.0/23
Deny from 163.61.74.0/23
Deny from 165.99.58.0/24
Deny from 165.99.59.0/24
Deny from 203.160.0.0/23
Deny from 203.160.132.0/22
Deny from 203.162.0.0/16
Deny from 203.210.128.0/17
Deny from 203.210.128.0/18
Deny from 203.210.128.0/20
Deny from 203.210.136.0/22
Deny from 203.210.144.0/20
Deny from 203.210.158.0/24
Deny from 203.210.160.0/20
Deny from 203.210.176.0/20
Deny from 203.210.192.0/18
Deny from 203.210.192.0/20
Deny from 203.210.192.0/24
Deny from 203.210.208.0/20
Deny from 203.210.224.0/20
Deny from 203.210.237.0/24
Deny from 203.210.240.0/20
Deny from 221.132.0.0/20
Deny from 221.132.16.0/24
Deny from 221.132.17.0/24
Deny from 221.132.18.0/24
Deny from 221.132.19.0/24
Deny from 221.132.20.0/24
Deny from 221.132.21.0/24
Deny from 221.132.22.0/24
Deny from 221.132.23.0/24
Deny from 221.132.24.0/24
Deny from 221.132.25.0/24
Deny from 221.132.26.0/24
Deny from 221.132.27.0/24
Deny from 221.132.28.0/24
Deny from 221.132.29.0/24
Deny from 221.132.30.0/23
Deny from 221.132.30.0/24
Deny from 221.132.31.0/24
Deny from 221.132.32.0/21
Deny from 221.132.32.0/22
Deny from 221.132.37.0/24
Deny from 221.132.38.0/24
Deny from 221.132.52.0/23
Deny from 221.132.54.0/23
Deny from 222.252.0.0/16
Deny from 222.253.0.0/16
Deny from 222.254.0.0/16
Deny from 222.255.0.0/16
Deny from 23.32.249.0/24
Deny from 23.48.52.0/22
Deny from 23.48.56.0/22
Deny from 23.53.208.0/22
Deny from 23.53.212.0/22
Deny from 43.239.220.0/23
Deny from 45.124.92.0/22
Deny from 45.254.32.0/22


[Rhindeer]

I don't have all my utilities at my disposal (out of town), but if you're basically down, it might be worth trying this.  It has not been cleaned & de-duped & checked against users, etc.  This ASN appears to cover a big hunk of Vietnam. 

Thank yooou! My forum was getting hammered by this same ASN so adding this list to htaccess has made my current guest list drop from 3000+ to 800 in just a couple minutes. (And it's still rapidly dropping.)

[a10]

I see tons in the 160~,170~,180~ range, South America, Brazil.
And those from Vietnam.

[shawnb61]

I see tons in the 160~,170~,180~ range, South America, Brazil.
And those from Vietnam.

If it's affecting your site, it's Pareto analysis time... 

Find the ASN that seems to be causing the biggest grief.  Do some checks to confirm you're not cutting off your valid users.  Add to .htaccess.

It's a never-ending game of whack-a-mole. 

Cleaning the lists...  I did perform some rudimentary cleaning of the list above...  I sorted it, removed the ipv6 entries.  Performed a quick manual de-dupe by looking for CIDRs ending in /16, then deleting everything with the same first two nodes.  E.g., in this example:
123.456.0.0/16
123.456.56.0/24
123.456.57.0/24

...it's only necessary to keep the first entry, all other 123.456.*.* entries can be deleted since the /16 entry includes all those.  A quick pass doing so greatly reduces the size of the list. 

[shawnb61]

One other item to consider...  I've made the following change to my forum and it has basically eliminated the weird spikes in guest counts.  Effectively giving them a buzz cut.  Same with the MySQL CPU usage spikes that can accompany them. 

I believe those come from crawlers that disallow cookies.  SMF, not finding a cookie, can't find a session, so creates a new session - which looks like a new visitor.  But with cookies disabled, that crawler will never find their prior session, i.e., it'll never be used.  So the next page load creates yet another session.  Rinse & repeat...  So...  Don't write them.  Although writing a session is a trivial & fast I/O, if you pile on a tremendous number of them, it bottlenecks and can cause issues.

This helps with crawlers that disallow cookies that are loading pages - i.e., even when not blocked by .htaccess.  Not a lot of bots behave like this (or you'd see these spikes every day), but the ones that do can cause problems.

In Sources/Sessions.php, near the top of the sessionWrite() function, add this:

Code Select Expand

// Don't bother writing the session if cookies are disabled; no way to retrieve it later
   if (empty($_COOKIE))
      return true;
It is one part of this PR: https://github.com/SimpleMachines/SMF/pull/8394

You will need to undo this change if that PR is ever merged & included in a future patch.

[Rhindeer]

@ShawnB ahhhh excellent! I'm going to add that code in when I get home and see how it goes! Thank you so much!

[a10]

...and suddenly they all left :O) Remains Microsoft and other more or less gentle bots.

[vbgamer45]

@shawnb61  this is great thanks for the tip.