Most Online Today Increase

[mcpheex3]

This is the second time this has happened in about a month. We typically have under 50 visitors per day. The most we have had online legitimately was 100. Today, it said Most Online Today: 544. I don't see anything in the logs that would account for this. Any suggestions as to why it is happening? Thanks!

Flourish Forum

[Shambles]

Hi Erica

If I were in your position I'd get into the Raw Access Logs (available in your cPanel, if you have that) and have a ganders at the IP addresses that have been connecting, plus what they have been clicking.

You may find the clicks are quite benign...

[drewactual]

my page was over 30k 'visitors' last night, and the standard 70ish (users) as it usually is.  most IP's chinese, but couldn't figure out what they were doing..

i run MPM Worker flavor of php- with FPM and not dynamically (i can't recall the setting's name, but it's fixed to a point- keeping at least 10 workers free up to 225 workers and based off that domain's resources- all on a dedicated server).  the 'standard' available workers are 125 usually and even during the heaviest traffic no more than 12 workers were... working?.... meaning: i've no clue what they were doing, but they were touching and then gone... i almost broke my all time record of 33k total, but it seems to have ebbed just below that this time.  i was watching those workers closely last night during the rush (college football site, and national championship was last night- i have a legit and well regarded poster from HK and don't want to block him else there would be a wholesale block on that entire range).

edited to add: TTL on idle workers is set to 5 seconds, which is to free up resources as i understand it? and which works great with a forum as folks 'click' and then read for a few minutes before 'clicking' again--- no need in keeping that door open for indefinite time, but it is set up to spawn children as needed.... i don't know if it's optimized, but.. it works for this particular site.  good enough, anyway. 

right now there are 26k 'visitors and a few over 60 users... and seems to be holding fine... however, i too would love to know what's happening if for nothing else just to know.

curiously, the 33k happened last year almost to the date. 

[a10]

Chinese ip's? Had to resort to permanent cn blocking in htaccess, if removed it may be peaceful for a few days, then starting again withh up to 10.000's of daily hits. Been ongoing for many months. Not one's usual bots imo, suspecting state directed, seeing so many cn universities, schools, official offices etc ip's.

See https://www.simplemachines.org/community/index.php?topic=570548

[drewactual]

yup.. i tracked that thread too...

thing about the chinese bots is they have no regard for robots.txt.... they'll scan until they crash the server- and then start all over as soon as it recovers.   i ran into this at the same time that thread was made---- and it was a calamity of errors........ explained:

cPanel had rolled out an update, and there was a bug with it that reset the MPM Workers to 'cPanel default' which was 10 i think it was... also defaulted the ttl and children spawns... and REALLY jacked me up with all the legit posters getting in line for workers... the page crawled for at least three days until it dawned on me to look at that (thinking THAT much traffic was capable of bring the server down?- shows what i know to be honest because i thought that exactly what was happening)... once i read the cPanel support bugs about that bug and corrected it the traffic from China seems of no consequence except making the page look to a layman (me) like it is a heckuva lot more popular than it is...

cfb51 dot com is the page i'm talking about... check it out right now and you'll see for yourself.  as said, after the cPanel fix (recovery) the traffic doesn't seem to harm anything.. before? it was an issue.   

[VJJohn]

About a week ago there was an increase of brute force bot attacks from Chinese IPs. Then in the past 2 days, there was suddenly a massive sharp increase from them, including DOS attacks. If you lookup the IPs in your "Who's Online" list, I'll bet you see a lot of Chinese IPs there. I also see increased traffic from LeaseWeb, a popular host for spammers & hackers. It came from one of their branches in Kansas, but the particular LeaseWeb account hitting us turned out to be Chinese-owned.

Maybe you can change your "Who's Online" list to several hours or 8 hours to temporarily get a better view of who's doing what. If you see a lot of "unknown action", failed login, & registration attempts, or a single IP range slamming many links, & post in a row, that's probably the bad bots.

I tried banning them, but in my ban logs, I saw they still were relentlessly slamming the sites. I ended up having to block them at the front door with htaccess.

It's odd, I have many sites & SMF forums on different servers, & many on the same server. About half of them were untouched by them, & half were getting slammed hard. Most of them had links to each other on them. It seems like they were ignoring my biggest, busiest sites & targeting the small sites, or the ones with they least traffic. Sites getting 800-10,000 hits per day were untouched, sites getting 200 or less per day suddenly got slammed with around 1500 to over 3,000 hits in a day from Chinese servers & LeaseWeb. Maybe they think the smaller sites are less maintained, or poorly managed, so are easier to break or crack.

Unless you depend on Chinese traffic, just block all Chinese IPs that exhibit bad behavior.

[drewactual]

i use cleantalk on the forum in question, and it seems to do a great job of filtering out the crud... before it i was spending an hour or so a day going through them.  it's not free but it's worth it.  no member has ever complained about difficulty registering. 

the flow out seems as abrupt as the escalation- i'm under 26k right now as opposed to literally 31k an hour ago (meaning the ones last night at this time are falling off the log). 

they don't seem to be causing any issue i can identify.. all error logs are clear, and no strange operations requested... what i usually see with a hacker is an operation such as config.cgi would be requested, or some variance of a config file they would hope not protected?  as i said, they touch a page and bounce.  the google analytics for page views and bounce rate seems to be the most impacted, but who really cares about that? it is usually around 10~11%, but is well over that after this barrage.  like, nearing 20%.

[mcpheex3]

Thank you for the good info and for sharing your similar experiences.

The strange thing was that there were no big increases in the guests listed under the activity from "active users." Just the typical number looking at various posts. I was able to access the Raw Access Logs (thanks @Shambles) and I think it confirmed the issue.

I did see in the Spider logs that MSN has had 30k - 60k+ page indexes every other day or so in the past couple of weeks. In the Raw Access logs, the repeat IP was indeed an MSN Bingbot IP address. Would this account for the "Most Online Today" count?

I have a very strict registration process (which some users complain about) but I have not had many erroneous or multiple registrations. I manually approve every registration, require three questions answered in relation to our topic, and they must answer the "Reason for Joining" with a specific reference to our topic. The majority do not have a problem. But sometimes they forget to clear the "this box must remain empty" and get the 'user is a spam' error. Then they get mad and email me. I set up a quick "how to register" video which is linked in the top right newsletter section. So far, this has been a good process.

I hope I am not missing something. I approve the first three posts of every new user (and they must answer the questions for their first three posts).

Stupid question - how can I see which pages they were accessing this morning? All I could see was there were 500+ page hits but no link to tell me which pages.

Also, if a guest shows this error, "Unable to verify referring url. Please go back and try again. ?action=login2" three or more times over a couple hours - is it someone trying to break in?

I have several legitimate members from China so don't want to block China in htaccess entirely.

@drewactual - I see what you are talking about on your site (cool btw). I like how your stats show users online today. Mine only shows users on in the past 10 minutes. You are way ahead of me. I only understood about half of what you said! 

[drewactual]

believe me i speak more from ignorance than knowledge, knowing only enough to be dangerous.

if you're encountering harmless robots i wouldn't worry a bit about it... if you want to get in front of it, google up the syntax in a robots.txt and enter in the page types you WANT indexed and the ones you don't... this will limit the amount of time and resources those robots consume... w/o a robots.txt you may have something as harmless as google or Bing methodically go through ALL your pages- which can take time, and also do silly things like 'attempt' to access a page only to be stiff armed by authentication and then reporting to google index an erroneous '4xx encountered while attempting index' which takes a long time to correct (not that it matters much, but google WILL impact your rankings based on indexing 'coverage' faults/errors...

the 'unable to verify referring URL' is most likely a cache on your users computers from prior to 2.0.14, so, rec they clear it out and get a fresh set of files... give it a shot and see if they hang around...

that's about all the advice i can offer except simply to don't fix what isn't broken... if this traffic isn't impacting function just let it go.  keep an eye on it but don't lose sleep over it until it really is a problem... 

[drewactual]

Just another piece of info that may be of value:  the night before last when the traffic above hit cleantalk firewall blocked 28k original IP's according to the weekly update they just sent me...

Off the top of my head I dont even remember how I set cleantalk up on the page.  And, this certainly isnt intended as a advert for them... What I'm thinking is the IPs simply matched known malicious IPs in their (or others that they use) databases, and they were blocked.

This explains the touching and then POOF gone traffic.  At some point I may dig into their code to see how they go about this... I know it costs me more than the fee (it costs the response time a few milliseconds) so even scouring the code may do me no good as it may be a function on their server not mine.  All the same, and no matter how it works, it does and thats all that matters.  There were no errirs logged by either SMF or the server during the event.

[mcpheex3]

That is of value - thank you. I am going to look into cleantalk. Thanks!