mysql "Got an error reading communication packets"

Started by sah62, October 24, 2019, 12:14:44 PM

Previous topic - Next topic

sah62

I get SMF database aborted connection warning messages like this fairly frequently in my mysql error log:

2019-10-23 15:31:42 12430 [Warning] Aborted connection 12430 to db: 'smf' user: 'user' host: 'localhost' (Got an error reading communication packets)

I have the mysql max_allowed_packet value set to 64M, but I really don't know if increasing that value is what's needed to address this issue. Any tips?

Illori

are you the host or do you have a host that manages your server?

sah62

I'm running the site on a Linode VM for which I have full shell access.

sah62

Another clue: I noticed this warning in my php-fpm log at the same time:


[29-Oct-2019 15:52:52] WARNING: [pool mypool] child 8629 exited on signal 7 (SIGBUS - core dumped) after 3197.032572 seconds from start


There's a corresponding warning in the mysql error log:


2019-10-29 15:52:52 31459 [Warning] Aborted connection 31459 to db: 'smf' user: 'user' host: 'localhost' (Got an error reading communication packets)


There's a php-fpm apport crash report file in /var/crash, but without symbols I can't see a meaningful stack trace.

sah62

More clues from my nginx log:


2019/10/29 15:52:52 [error] 1244#1244: *105640 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 155.178.180.11, server: mysite.org, request: "GET /forum/index.php?action=dlattach;topic=26837.0;attach=18821;image HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.2-fpm-mysite.sock:", host: "www.mysite.org", referrer: "https://www.mysite.org/forum/index.php?topic=26837.30"


and


155.178.180.11 - - [29/Oct/2019:15:52:52 -0400] "GET /forum/index.php?action=dlattach;topic=26837.0;attach=18821;image HTTP/1.1" 502 584 "https://www.mysite.org/forum/index.php?topic=26837.30" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.75 Safari/537.36"


Aha, a 502 error! The last answer found on this page might be helpful:

https://serverfault.com/questions/543999/nginx-errors-recv-failed-104-connection-reset-by-peer-while-reading-respon/782128

I had the memory_limit variable set to 128M in my php-fpm configuration. I just tried upping it to 256M. We'll see if that makes a difference.

sah62

Upping memory_limit to 256M didn't help. Setting output_buffering = On seems to be doing the trick, though. There haven't been any warnings for the past several hours.

https://stackoverflow.com/questions/22697584/nginx-uwsgi-104-connection-reset-by-peer-while-reading-response-header-from-u

There may be some optimal value to use that's less than the "no limit" you get with "On" (the default is 4096 bytes), but I haven't done any experimentation to see what will work in my environment.

sah62

I'm still seeing these 502 errors every so often. They all appear to be related to image processing. More examples:


<address> - - [13/Jan/2020:10:45:23 -0500] "GET /forum/index.php?action=dlattach;topic=27033.0;attach=19331;image HTTP/1.1" 502 584 "https://www.mysite.org/forum/index.php?topic=27033.0" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363"
<address> - - [13/Jan/2020:12:58:51 -0500] "GET /forum/index.php?action=dlattach;attach=8516;type=avatar HTTP/1.1" 502 584 "https://www.mysite.org/forum/index.php?topic=26983.0" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"
<address> - - [13/Jan/2020:14:10:43 -0500] "GET /forum/index.php?action=dlattach;topic=27033.0;attach=19337;image HTTP/1.1" 502 584 "https://www.mysite.org/forum/index.php?topic=27033.0" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
<address> - - [13/Jan/2020:14:52:14 -0500] "GET /forum/index.php?action=dlattach;attach=10808;type=avatar HTTP/1.1" 502 584 "https://www.mysite.org/forum/index.php?topic=27028.0;topicseen" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"
<address> - - [13/Jan/2020:16:20:39 -0500] "GET /forum/index.php?action=dlattach;attach=15181;type=avatar HTTP/1.1" 502 182 "https://www.mysite.org/forum/index.php?topic=27028.0" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko"
<address> - - [13/Jan/2020:18:38:41 -0500] "GET /forum/index.php?action=dlattach;topic=27032.0;attach=19345;image HTTP/1.1" 502 584 "https://www.mysite.org/forum/index.php?topic=27032.0" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"
<address> - - [13/Jan/2020:19:27:10 -0500] "GET /forum/index.php?action=dlattach;topic=27058.0;attach=19393;image HTTP/1.1" 502 182 "https://www.mysite.org/forum/index.php?topic=27058.0" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
<address> - - [13/Jan/2020:22:00:47 -0500] "GET /forum/index.php?action=dlattach;topic=27044.0;attach=19355;image HTTP/1.1" 502 182 "https://www.mysite.org/forum/index.php?topic=27044.0" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15) AppleWebKit/605.1.15 (KHTML, like Gecko) FxiOS/21.0 Version/13.0.3 Safari/605.1.15"
<address> - - [14/Jan/2020:00:16:21 -0500] "GET /forum/index.php?action=dlattach;attach=10809;type=avatar HTTP/1.1" 502 584 "https://www.mysite.org/forum/index.php?topic=24730.105" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363"
<address> - - [14/Jan/2020:00:16:21 -0500] "GET /forum/index.php?action=dlattach;topic=24730.0;attach=15630;image HTTP/1.1" 502 584 "https://www.mysite.org/forum/index.php?topic=24730.105" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363"
<address> - - [14/Jan/2020:09:56:33 -0500] "GET /forum/index.php?PHPSESSID=94u8rk5ttviq6fv7abnhu9rmdl&action=dlattach;topic=27058.0;attach=19389;image HTTP/1.1" 502 584 "https://www.mysite.org/forum/index.php?PHPSESSID=94u8rk5ttviq6fv7abnhu9rmdl&topic=27058.0" "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; Win64; x64; Trident/6.0; MDDCJS)"


Note the action passed to index.php: "action=dlattach", with each error associated with an avatar or an image attached to a topic. Is there a clue here?

Illori


sah62

Quote from: Illori on January 14, 2020, 10:29:53 AM
which version of SMF are you using?

2.0.17 with PHP 7.2 on Ubuntu 18.04.3 with nginx/1.14.0.

vbgamer45

I would suggest this optimization
https://www.simplemachines.org/community/index.php?topic=293441.0
#1 move avatars to a custom location and then move existing ones as well
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

sah62

Quote from: vbgamer45 on January 14, 2020, 11:26:16 AM
I would suggest this optimization
https://www.simplemachines.org/community/index.php?topic=293441.0
#1 move avatars to a custom location and then move existing ones as well

OK, done, thanks for the tip. I'll let things run for a bit to see what happens.

sah62

Quote from: sah62 on January 14, 2020, 11:43:33 AM
Quote from: vbgamer45 on January 14, 2020, 11:26:16 AM
I would suggest this optimization
https://www.simplemachines.org/community/index.php?topic=293441.0
#1 move avatars to a custom location and then move existing ones as well

OK, done, thanks for the tip. I'll let things run for a bit to see what happens.

Approximately 22 hours of run time and I haven't seen any more errors. Thanks!

Advertisement: