BingBot causing Apache webserver to hang

[MobileCS]

BingBot is trying to access the forum and causing issues. I have never seen this before.

All 50 of my available Apache process are at 50%+ load and my error log looks like this :

Code Select Expand

[Thu Nov 21 08:47:41.512104 2019] [php7:error] [pid 75699] [client 40.77.167.157:0] PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /usr/opt/www/example.com/httpdocs/forum/Sources/Load.php on line 239
[Thu Nov 21 08:47:42.129128 2019] [php7:error] [pid 75695] [client 207.46.13.175:0] PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /usr/opt/www/example.com/httpdocs/forum/Sources/Load.php on line 262
[Thu Nov 21 08:47:43.463125 2019] [php7:error] [pid 75707] [client 40.77.167.157:0] PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /usr/opt/www/example.com/httpdocs/forum/Sources/Load.php on line 239
[Thu Nov 21 08:47:45.213124 2019] [php7:error] [pid 75706] [client 207.46.13.75:0] PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /usr/opt/www/example.com/httpdocs/forum/Sources/Load.php on line 262
[Thu Nov 21 08:47:51.641110 2019] [php7:error] [pid 75719] [client 40.77.167.157:0] PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /usr/opt/www/example.com/httpdocs/forum/Sources/Load.php on line 262

I have plenty of ram to spare, and restarting Apache solves nothing. It's only Microsoft that is making the Apache process hang like this.

Code Select Expand


last pid: 76172;  load averages: 27.90, 29.27, 35.65
94 processes:  34 running, 60 sleeping
CPU:  100% user,  0.0% nice,  0.0% system,  0.0% interrupt,  0.0% idle
Mem: 1408M Active, 8908M Inact, 2262M Wired, 1490M Buf, 4433M Free
Swap: 3881M Total, 3881M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
76168 www           1  45    0   178M    48M CPU2     2   0:02 100.09% httpd
75722 www           1  79    0   182M    58M RUN      6   1:56  32.22% httpd
75758 www           1  79    0   181M    59M CPU1     1   1:46  27.65% httpd
76138 www           1  79    0   177M    47M RUN      4   0:09  27.49% httpd
75697 www           1  79    0   181M    55M RUN      1   2:10  27.38% httpd
75723 www           1  78    0   182M    55M CPU7     7   1:54  27.13% httpd
75707 www           1  79    0   178M    50M RUN      1   2:11  26.27% httpd
76034 www           1  78    0   178M    53M RUN      7   0:33  26.26% httpd
75712 www           1  80    0   182M    55M RUN      6   1:52  26.11% httpd
75912 www           1  78    0   181M    54M RUN      3   1:04  25.84% httpd
76094 www           1  78    0   187M    57M RUN      3   0:19  25.84% httpd
75699 www           1  78    0   181M    59M RUN      0   2:10  25.73% httpd
75921 www           1  80    0   182M    56M CPU4     4   1:00  24.91% httpd
75978 www           1  78    0   178M    50M RUN      4   0:54  24.70% httpd
75717 www           1  78    0   182M    59M RUN      3   1:57  24.55% httpd
75970 www           1  79    0   182M    60M CPU3     3   0:49  23.90% httpd
75913 www           1  78    0   182M    60M RUN      7   1:04  23.79% httpd
75986 www           1  79    0   182M    54M RUN      0   0:52  23.59% httpd
75724 www           1  79    0   182M    58M RUN      5   2:00  23.32% httpd
75719 www           1  79    0   182M    57M CPU5     5   1:55  23.21% httpd
76134 www           1  78    0   178M    53M RUN      5   0:08  23.11% httpd
76116 www           1  79    0   181M    59M RUN      4   0:12  22.99% httpd
75888 www           1  78    0   181M    59M RUN      7   1:15  22.93% httpd
75713 www           1  79    0   181M    59M RUN      0   1:42  22.47% httpd
76141 www           1  79    0   181M    52M RUN      6   0:07  21.31% httpd
75911 www           1  78    0   181M    54M RUN      5   1:03  20.56% httpd
75762 www           1  79    0   182M    60M RUN      6   1:55  20.23% httpd
75878 www           1  79    0   178M    53M CPU0     0   1:15  19.19% httpd
75983 www           1  79    0   182M    59M RUN      5   0:50  18.89% httpd
75706 www           1  79    0   182M    59M RUN      1   2:06  18.82% httpd

[MobileCS]

All the requests are trying to access the forum main page :

Code Select Expand

/forum/index.php?PHPSESSID=d2c2e41982bb4b980308f2d867918891
/forum/index.php?PHPSESSID=da08a05dabcde04bc28ec5cb13d90fca
/forum/index.php?PHPSESSID=164ecfb38ab9d9bc056e2b7adc1527e8
/forum/index.php?PHPSESSID=8adb0aeae0718fa7f493acb12ed2d11f
/forum/index.php?PHPSESSID=6016ed5108e66df425cb200363e2e72c

My webserver has handled 20x the traffic I am getting now and did not have this issue.

[MobileCS]

I managed to solve the issue by reverting back to PHP 7.1 and removing the SMF Hacks PHP 7.2+ compatibility mod.

If anyone else is using that mod, check your error logs as it will make your forum fail to load for random people.

I really hope SMF 2.0.16 will be available soon, as I will not be able to stay on this PHP version for long.