News:

Want to get involved in developing SMF, then why not lend a hand on our github!

Main Menu

SMF 2.0.x Causing High Loads and Server to Crash?

Started by statusjpn, December 14, 2014, 04:24:53 AM

Previous topic - Next topic

statusjpn

Hello there,

Stuck with a problem that i hope you guys can help me with. I have a managed dedicated server with a local hosting company that i have been hosting a few sites (3 x IPB, 3 x WP, 1 x SMF) for the last 2 years with no issues.

Just a couple of weeks ago, i noticed that the server started to experience extremely high loads and timing out and i would have to write in to the data-centre to request a hard reboot to the server (4 times in the last week alone). The hosting company has kindly indicated that the account hosting the SMF site was the culprit and shared the logs below with me.

I have an 8 core server and the load spikes were up to as high as 20 and from the site takes >1 minute to load.

I am running Redhat with cPanel WHM and PHP version is 3.3 and apache version 2.2. MySQL is the latest version.

I had initially thought that the updating SMF (i was running 2.0.7 before yesterday) would solve the problem as i saw in the changelog that there were some issues with PHP compatibility but the problem came back again after ~12 hours.

Not sure where to go from here and i have suspended the SMF account temporarily so that my other sites can operate normally and server's load is back to ~4.

I would deeply appreciate it if someone looked at the logs below to let me know how i can fix the issue or would updated PHP to version 3.4 help at all?

Thanks in advance! Cheers!



Responsible for the crash: suzukicl

%----cmdline
# Command line
/usr/bin/php /home/suzukicl/public_html/index.php

%----reason
# Reason of crash
Process /usr/bin/php was killed by signal 7 (SIGBUS)

Crashes:
@0
Directory: /var/spool/abrt/ccpp-2014-12-13-11:45:45-1143
executable: /usr/bin/php
time: Sat 13 Dec 2014 11:47:09 AM SGT
uid: 504

@1
Directory: /var/spool/abrt/ccpp-2014-12-12-00:28:26-15195
executable: /usr/bin/php
time: Fri 12 Dec 2014 12:28:49 AM SGT
uid: 504

@2
Directory: /var/spool/abrt/ccpp-2014-12-12-00:26:38-15404
executable: /usr/bin/php
time: Fri 12 Dec 2014 12:26:38 AM SGT
uid: 504

@3
Directory: /var/spool/abrt/ccpp-2014-12-12-00:25:41-15279
executable: /usr/bin/php
time: Fri 12 Dec 2014 12:26:38 AM SGT
uid: 504

@4
Directory: /var/spool/abrt/ccpp-2014-12-10-04:46:00-9715.new
executable: /usr/bin/php
time: Wed 10 Dec 2014 04:55:30 AM SGT
uid: 504

@5
Directory: /var/spool/abrt/ccpp-2014-12-10-02:37:46-9318
executable: /usr/bin/php
time: Wed 10 Dec 2014 02:40:04 AM SGT
uid: 504

@6
Directory: /var/spool/abrt/ccpp-2014-12-10-02:05:14-7855
executable: /usr/bin/php
time: Wed 10 Dec 2014 02:08:10 AM SGT
uid: 504

@7
Directory: /var/spool/abrt/ccpp-2014-12-10-01:10:56-9684
executable: /usr/bin/php
time: Wed 10 Dec 2014 01:12:45 AM SGT
uid: 504

@8
Directory: /var/spool/abrt/ccpp-2014-12-09-21:02:31-6449
executable: /usr/bin/php
time: Tue 09 Dec 2014 09:02:33 PM SGT
uid: 504

@9
Directory: /var/spool/abrt/ccpp-2014-12-09-21:02:31-6498
executable: /usr/bin/php
time: Tue 09 Dec 2014 09:02:33 PM SGT
uid: 504

@10
Directory: /var/spool/abrt/ccpp-2014-12-09-21:01:17-6532
executable: /usr/bin/php
time: Tue 09 Dec 2014 09:01:17 PM SGT
uid: 504

@11
Directory: /var/spool/abrt/ccpp-2014-12-09-20:35:02-5776
executable: /usr/bin/php
time: Tue 09 Dec 2014 08:35:02 PM SGT
uid: 504

@12
Directory: /var/spool/abrt/ccpp-2014-12-09-20:34:35-5764
executable: /usr/bin/php
time: Tue 09 Dec 2014 08:34:35 PM SGT
uid: 504



[New Thread 1143]
[Thread debugging using libthread_db enabled]
Core was generated by `/usr/bin/php /home/suzukicl/public_html/index.php'.
Program terminated with signal 7, Bus error.
\#0 0x000000365da8967e in memcpy () from /lib64/libc.so.6

Thread 1 (Thread 0x7f19adc6d7e0 (LWP 1143)):
\#0 0x000000365da8967e in memcpy () from /lib64/libc.so.6
No symbol table info available.
\#1 0x00007f19ada0b5b7 in ?? () from /usr/local/IonCube/ioncube_loader_lin_5.3.so
No symbol table info available.
\#2 0x00007f19ada90423 in ?? () from /usr/local/IonCube/ioncube_loader_lin_5.3.so
No symbol table info available.
\#3 0x00007f19ada93e0a in ?? () from /usr/local/IonCube/ioncube_loader_lin_5.3.so
No symbol table info available.
\#4 0x00007f19ada96cb1 in _zval_dup () from /usr/local/IonCube/ioncube_loader_lin_5.3.so
No symbol table info available.
\#5 0x00007f19ada14c93 in ?? () from /usr/local/IonCube/ioncube_loader_lin_5.3.so
No symbol table info available.
\#6 0x000000000066926e in compile_filename (type=8, filename=0x7f19ad3042a0) at Zend/zend_language_scanner.l:407
file_handle = {type = ZEND_HANDLE_FILENAME, filename = 0x209b6c8 "/home/suzukicl/public_html/cache/data_d58456c426f4160d02b0def2d607192b-SMF-theme_settings-4.php", opened_path = 0x0, handl$
tmp = {value = {lval = 0, dval = 0, str = {val = 0x0, len = 36230256}, ht = 0x0, obj = {handle = 0, handlers = 0x228d470}}, refcount__gc = 36252576, type = 1 '\001', is_ref__gc = 0 '\000'}
retval = <value optimized out>
\#7 0x00000000006bebf3 in ZEND_INCLUDE_OR_EVAL_SPEC_TMP_HANDLER (execute_data=0x7f19ad3035f8) at /home/cpeasyapache/src/php-5.3.29/Zend/zend_vm_execute.h:5257
opline = 0x7f19ad1b5b60
new_op_array = 0x0
return_value_used = 0
free_op1 = {var = 0x7f19ad3042a0}
inc_filename = 0x7f19ad3042a0
tmp_inc_filename = 0x0
failure_retval = 0 '\000'
\#8 0x00000000006b39f0 in execute (op_array=0x2021850) at /home/cpeasyapache/src/php-5.3.29/Zend/zend_vm_execute.h:107
ret = <value optimized out>
execute_data = 0x7f19ad3035f8
nested = 1 '\001'
original_in_execution = 1 '\001'
\#9 0x00007f19ad80eaa6 in zend_oe () from /usr/local/Zend/lib/Guard-5.5.0/php-5.3.x/ZendGuardLoader.so
No symbol table info available.
\#10 0x00000000006dbce6 in zend_do_fcall_common_helper_SPEC (execute_data=<value optimized out>) at /home/cpeasyapache/src/php-5.3.29/Zend/zend_vm_execute.h:349
opline = 0x7f19ad0f8ed0
should_change_scope = 1 '\001'
\#11 0x00000000006b39f0 in execute (op_array=0x20200b0) at /home/cpeasyapache/src/php-5.3.29/Zend/zend_vm_execute.h:107
ret = <value optimized out>
execute_data = 0x7f19ad2fa660
nested = 1 '\001'
original_in_execution = 1 '\001'
\#12 0x00007f19ad80eaa6 in zend_oe () from /usr/local/Zend/lib/Guard-5.5.0/php-5.3.x/ZendGuardLoader.so
No symbol table info available.
\#13 0x00000000006dbce6 in zend_do_fcall_common_helper_SPEC (execute_data=<value optimized out>) at /home/cpeasyapache/src/php-5.3.29/Zend/zend_vm_execute.h:349
opline = 0x1edc930
should_change_scope = 1 '\001'
\#14 0x00000000006b39f0 in execute (op_array=0x1f1a5e0) at /home/cpeasyapache/src/php-5.3.29/Zend/zend_vm_execute.h:107
ret = <value optimized out>
execute_data = 0x7f19ad2f80a8
nested = 1 '\001'
original_in_execution = 1 '\001'
\#15 0x00007f19ad80eaa6 in zend_oe () from /usr/local/Zend/lib/Guard-5.5.0/php-5.3.x/ZendGuardLoader.so
No symbol table info available.
\#16 0x00000000006dbce6 in zend_do_fcall_common_helper_SPEC (execute_data=<value optimized out>) at /home/cpeasyapache/src/php-5.3.29/Zend/zend_vm_execute.h:349
opline = 0x1ebecd0
should_change_scope = 1 '\001'
\#17 0x00000000006b39f0 in execute (op_array=0x1eb22b0) at /home/cpeasyapache/src/php-5.3.29/Zend/zend_vm_execute.h:107
ret = <value optimized out>
execute_data = 0x7f19ad2f6050
nested = 1 '\001'
original_in_execution = 0 '\000'
\#18 0x00007f19ad80eaa6 in zend_oe () from /usr/local/Zend/lib/Guard-5.5.0/php-5.3.x/ZendGuardLoader.so
No symbol table info available.
\#19 0x000000000068d4ff in zend_execute_scripts (type=8, retval=0x0, file_count=3) at /home/cpeasyapache/src/php-5.3.29/Zend/zend.c:1259
files = {{gp_offset = 40, fp_offset = 0, overflow_arg_area = 0x7fff33ddbcd0, reg_save_area = 0x7fff33ddbc60}}
i = <value optimized out>
file_handle = 0x7fff33de0350
orig_op_array = 0x0
orig_retval_ptr_ptr = 0x0
\#20 0x000000000063a6d7 in php_execute_script (primary_file=0x7fff33de0350) at /home/cpeasyapache/src/php-5.3.29/main/main.c:2316
realfile = "ublic_html/index.php", '\000' <repeats 252 times>, "\002", '\000' <repeats 23 times>, "@", '\000' <repeats 31 times>, "\002\000\000\000\060\000\000\000[\000\000\000|\000\000\00$
__orig_bailout = 0x7fff33de0140
__bailout = {{__jmpbuf = {13742016, -3216994030098228173, 140734063582770, 30156032, 1, 140734063582721, -3216994029095789517, 3216862634037435443}, __mask_was_saved = 0, __saved_mask = {_$
prepend_file_p = <value optimized out>
append_file_p = 0x0
prepend_file = {type = ZEND_HANDLE_FILENAME, filename = 0x0, opened_path = 0x0, handle = {fd = 0, fp = 0x0, stream = {handle = 0x0, isatty = 0, mmap = {len = 0, pos = 0, map = 0x0, buf = 0$
append_file = {type = ZEND_HANDLE_FILENAME, filename = 0x0, opened_path = 0x0, handle = {fd = 0, fp = 0x0, stream = {handle = 0x0, isatty = 0, mmap = {len = 0, pos = 0, map = 0x0, buf = 0x$
old_cwd = 0x7fff33ddbce0 "/home/suzukicl/public_html"
use_heap = 0 '\000'
retval = 0
\#21 0x0000000000717a3e in main (argc=2, argv=0x7fff33de05d8) at /home/cpeasyapache/src/php-5.3.29/sapi/cgi/cgi_main.c:2135
__orig_bailout = 0x0
__bailout = {{__jmpbuf = {0, 3216861907169401907, 13716608, 13422048, 0, 0, -3216994030146462669, 3216862748340083763}, __mask_was_saved = 0, __saved_mask = {__val = {0, 0, 0, 0, 0, 0, 0, $
free_query_string = 0
exit_status = 0
cgi = 1
c = <value optimized out>
i = <value optimized out>
len = <value optimized out>
file_handle = {type = ZEND_HANDLE_MAPPED, filename = 0x1e9c160 "/home/suzukicl/public_html/index.php", opened_path = 0x0, handle = {fd = 32187872, fp = 0x1eb25e0, stream = {handle = 0x1eb2$
s = <value optimized out>
behavior = 1
no_headers = 0
orig_optind = 1
orig_optarg = 0x0
script_file = 0x0
ini_entries_len = <value optimized out>
max_requests = 500
requests = 0
fastcgi = 0
bindpath = <value optimized out>
fcgi_fd = <value optimized out>
request = {listen_socket = -1373425664, fd = 32537, id = -1375922992, keep = 32537, closed = 1568803224, in_len = 54, in_pad = -1379446784, out_hdr = 0x0, out_pos = 0x1 <Address 0x1 out of$
repeats = 1
benchmark = <value optimized out>
start = {tv_sec = 0, tv_usec = 4402651}
end = {tv_sec = 140733193388033, tv_usec = 7872000}
status = 0
query_string = <value optimized out>
decoded_query_string = <value optimized out>
skip_getopt = <value optimized out>

LiroyvH

Any mods? How big is the forum? What kind of database are you using? Did you optimize mySQL? Do you have sufficient memory available; eg when you see the issue occurring: is there enough free memory, or is the server using swap memory like a maniac...?

In other words: more details pls? :)
((U + C + I)x(10 − S)) / 20xAx1 / (1 − sin(F / 10))
President/CEO of Simple Machines - Server Manager
Please do not PM for support - anything else is usually OK.

statusjpn

You're spot on. There was no more free memory and i was maxing out the 4Gb of ram. Did an upgrade to 8Gb and all is good once again.

Thank you for your input.

Btw for those that are wonder, i did the "free -m" command through SSH to see what was up with the RAM usage.

Cheers!

LiroyvH

((U + C + I)x(10 − S)) / 20xAx1 / (1 − sin(F / 10))
President/CEO of Simple Machines - Server Manager
Please do not PM for support - anything else is usually OK.

Advertisement: