News:

Bored?  Looking to kill some time?  Want to chat with other SMF users?  Join us in IRC chat or Discord

Main Menu

SMF Crash - Can Someone Help ID The Cause?

Started by sah62, September 12, 2022, 01:26:52 PM

Previous topic - Next topic

sah62

I'm seeing intermittent errors in multiple logs from a php-fpm crash. I don't think this is a SMF error, but I'm hoping that someone can help me understand what I'm seeing and possibly help figure out what's crashing. I'm running SMF 2.1.2, PHP 8.1.2, nginx 1.18.0, MariaDB 10.9.2, Ubuntu 22.04.1 LTS.

Starting at the top, this is what I see in my nginx access and error logs:

96.60.12.199 - - [12/Sep/2022:11:50:46 -0400] "GET /smf/index.php?board=36.0 HTTP/1.1" 502 552 "https://www.mysite.com/smf/index.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 Edg/105.0.1343.33"

2022/09/12 11:50:46 [error] 1176207#1176207: *451663 recv() failed (104: Unknown error) while reading response header from upstream, client: 96.60.12.199, server: www.mysite.com, request: "GET /smf/index.php?board=36.0 HTTP/1.1", upstream: "fastcgi://unix:/run/php/php-fpm-smf.sock:", host: "www.mysite.com", referrer: "https://www.mysite.com/smf/index.php"

MariaDB log:

2022-09-12 11:50:46 11303 [Warning] Aborted connection 11303 to db: 'smf' user: 'smf' host: 'localhost' (Got an error reading communication packets)
PHP-FPM log:

[12-Sep-2022 11:50:46] WARNING: [pool smf] child 1392103 exited on signal 7 (SIGBUS - core dumped) after 25.025117 seconds from start
Info from the apport crash file:

$ sudo apport-retrace --rebuild-package-info -g /var/crash/_usr_sbin_php-fpm8.1.0.crash
GNU gdb (Ubuntu 12.0.90-0ubuntu1) 12.0.90
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Reading symbols from /usr/sbin/php-fpm8.1...
(No debugging symbols found in /usr/sbin/php-fpm8.1)
warning: Can't open file /dev/zero (deleted) during file-backed mapping note processing
warning: Can't open file /var/www/mysite/smf/attachments/45602_bee708091e2657a16f0067da3560ffefd2a68587.dat_thumb.tmp during file-backed mapping note processing
[New LWP 1392103]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `php-fpm: pool smf                                                           '.
Program terminated with signal SIGBUS, Bus error.
#0  __memcpy_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:103
103     ../sysdeps/x86_64/multiarch/memcpy-ssse3.S: No such file or directory.
(gdb) bt
#0  __memcpy_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:103
#1  0x00007fc221ee4fae in ReadBlob () from /lib/x86_64-linux-gnu/libMagickCore-6.Q16.so.6
#2  0x00007fc21ee8c763 in ?? () from /usr/lib/x86_64-linux-gnu/ImageMagick-6.9.11/modules-Q16/coders/jpeg.so
#3  0x00007fc2226aebc0 in jpeg_fill_bit_buffer () from /lib/x86_64-linux-gnu/libjpeg.so.8
#4  0x00007fc2226af442 in ?? () from /lib/x86_64-linux-gnu/libjpeg.so.8
#5  0x00007fc2226a8e16 in ?? () from /lib/x86_64-linux-gnu/libjpeg.so.8
#6  0x00007fc2226ae8d2 in ?? () from /lib/x86_64-linux-gnu/libjpeg.so.8
#7  0x00007fc2226a6225 in jpeg_read_scanlines () from /lib/x86_64-linux-gnu/libjpeg.so.8
#8  0x00007fc21ee8dbab in ?? () from /usr/lib/x86_64-linux-gnu/ImageMagick-6.9.11/modules-Q16/coders/jpeg.so
#9  0x00007fc21ee8e179 in ?? () from /usr/lib/x86_64-linux-gnu/ImageMagick-6.9.11/modules-Q16/coders/jpeg.so
#10 0x00007fc221f1d471 in ReadImage () from /lib/x86_64-linux-gnu/libMagickCore-6.Q16.so.6
#11 0x00007fc22220cc7e in MagickReadImage () from /lib/x86_64-linux-gnu/libMagickWand-6.Q16.so.6
#12 0x00007fc22232255d in php_imagick_read_file () from /usr/lib/php/20210902/imagick.so
#13 0x00007fc2223289c5 in zim_Imagick___construct () from /usr/lib/php/20210902/imagick.so
#14 0x0000557bfa6d4563 in execute_ex ()
#15 0x0000557bfa6d6429 in zend_execute ()
#16 0x0000557bfa666ef0 in zend_execute_scripts ()
#17 0x0000557bfa6024ca in php_execute_script ()
#18 0x0000557bfa4ac1e2 in ?? ()
#19 0x00007fc2266c8d90 in __libc_start_call_main ([email protected]=0x557bfa4ab2a0, [email protected]=4, [email protected]=0x7ffd7e68c068)
    at ../sysdeps/nptl/libc_start_call_main.h:58
#20 0x00007fc2266c8e40 in __libc_start_main_impl (main=0x557bfa4ab2a0, argc=4, argv=0x7ffd7e68c068, init=<optimized out>,
    fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd7e68c058) at ../csu/libc-start.c:392
#21 0x0000557bfa4adeb5 in _start ()
(gdb)

I'm seeing these errors on two different forums on two different servers running the same code as described above. Am I correct in thinking that this appears to be a crash when the imagick library attempts to do a memcpy? If so, what might cause that besides an imagick bug?

Steve

Please do not PM me for support.

sah62


Doug Heffernan

Quote from: sah62 on September 12, 2022, 01:26:52 PM451663 recv() failed (104: Unknown error) while reading response header from upstream, client: 96.60.12.199

This indictes that upstream server had closed the TCP connection while APISIX reading response header.

You should extend the timeout period of the upstream services to be greater than the timeouts configured in APISIX.

Quote from: sah62 on September 12, 2022, 01:26:52 PM2022-09-12 11:50:46 11303 [Warning] Aborted connection 11303 to db: 'smf' user: 'smf' host: 'localhost' (Got an error reading communication packets)

This warning is triggered whenever the status counter for the aborted_clients or aborted_connects metric values are incremented.

The above quoted errors, and the other 2, have nothing to do with Smf. I.e. there is nothing that we can do on our end. You must talk to your host about this.



sah62

I don't think this is just a simple timeout situation. Note the memcpy fault in the stack trace.

I caught a slightly different error today that's directly related to an SMF operation:

2022/09/22 15:34:04 [error] 723952#723952: *1769 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Uncaught ImagickException: no decode delegate for this image format `TMP' @ error/constitute.c/ReadImage/575 in /var/www/mysite/web/forum/Sources/Subs-Graphics.php:452
Stack trace:
#0 /var/www/mysite/web/forum/Sources/Subs-Graphics.php(452): Imagick->__construct()
#1 /var/www/mysite/web/forum/Sources/Subs-Graphics.php(398): resizeImage()
#2 /var/www/mysite/web/forum/Sources/Subs-Graphics.php(147): resizeImageFile()
#3 /var/www/mysite/web/forum/Sources/Subs-Attachments.php(1237): createThumbnail()
#4 /var/www/mysite/web/forum/Sources/Subs-Attachments.php(1049): loadAttachmentContext()
#5 /var/www/mysite/web/forum/Sources/Subs.php(1746): parseAttachBBC()
#6 /var/www/mysite/web/forum/Sources/Subs.php(3636): {closure}()
#7 /var/www/mysite/web/forum/Sources/MessageIndex.php(395): parse_bbc()
#8 /var/www/mysite/web/forum/index.php(191): MessageIndex()
#9 {main}
  thrown in /var/www/mysite/web/forum/Sources/Subs-Graphics.php on line 452" while reading response header from upstream, client: 1.1.1.1, server: www.mysite.org, request: "GET /forum/index.php?board=8.0 HTTP/1.1", upstream: "fastcgi://unix:/run/php/php-fpm-smf.sock:", host: "www.mysite.org", referrer: "https://www.mysite.org/forum/index.php"


Kindred

Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

Advertisement: