Advertisement:

Author Topic: Huge amount of activity from China  (Read 2065 times)

Offline memiller

  • Semi-Newbie
  • *
  • Posts: 22
Re: Huge amount of activity from China
« Reply #40 on: December 06, 2019, 06:27:23 PM »
Anecdotally, we just got a significant burst of traffic from IPs in Huawei Clouds prefixes that seemed be indexing our whole site.  We use Cloudflare, so I added a firewall rule there to captcha challenge everything from AS136907 just for giggles. We got 18K hits in less than 6 hours from hundreds of IPs in the 159.138.144.0/20 (Hong Kong) range.  That's a hullava indexing run, if that's what it is. Honestly, I don't mind SEO and spider runs to help keep us discoverable, but it would be nice if it didn't include getting mugged.  :)

Offline Antechinus

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 24,854
  • Master of BBC Abuse
Re: Huge amount of activity from China
« Reply #41 on: December 06, 2019, 06:31:35 PM »
That's normal Chinese bot behaviour. When they decide to index a site, they just throw stacks of bots at it and attempt to index everything as fast as they can. They don't care about the effects on your site. If they crash your server, they don't care. They're just after all your information, and will grab it if its available.

Offline aegersz

  • SMF Hero
  • ******
  • Posts: 1,543
  • Gender: Male
  • the "mods and tweaks" junkie
    • dopetalk
Re: Huge amount of activity from China
« Reply #42 on: January 16, 2020, 11:38:38 AM »
I am also experiencing this for the IP range: 114.119.128.107 ~ 114.119.167.95.

IP lookup says:

Country:China
Region:Guangdong
City:Shenzhen

Why do they do this indexing and how does it help anybody ?
The configuration of my Linux VPS (SMF 2.0 with 145 mods & some assorted manual tweaks) can be found here and notes on my mods can be found here (warning: those links will take you to a drug related forum). My (House) music DJ dedication page is here

Offline drewactual

  • Sr. Member
  • ****
  • Posts: 715
    • College Football Fan Site CFB51
Re: Huge amount of activity from China
« Reply #43 on: January 16, 2020, 01:17:48 PM »
the best i can tell it's nothing more than the same type of indexing google, bing, yahoo, ect. perform... it's just a lot more aggressive and they could care less what you've 'limited' with robots.txt.... they'll hit your server's resources with all of their resources and drill until it's done- if your server crashes they'll be sitting there waiting until it recovers and it starts all over again.

this next bit is somewhat unrelated but something i discovered and shared in another thread that may help someone?

I run a dedicated server with centos and apache- and with an MPM Worker configuration and FPM over top of it.  the httpd.conf is NOT configured to dynamic (i can't recall the command it is set to) but it maintains x number of workers in reserve, and expands and withdraws the available pool depending on load.  I had mine set to 125 workers with a possible 225 iirc workers, the ability to spawn children, and a 5 second TTL window.... problem i had to find out about and while under stress from these chinese bots: a recent cPanel update reset the worker function to default- and default is slight... 10 workers i think it was, and unlimited TTL... so.. resources were clogged with real members 'getting in line' for actions..... the 'bug' is documented on the cPanel forums... once i discovered this and altered the settings to where they were, no more issues... on Monday night i had over 30k 'visitors' on the site and it had zero impact on function or load time. 

i share this for folks who may be running MPM Worker and are getting hit with this traffic, and who are experiencing pages that crawl... to simply look into it or to ask their hosts to do so...