News:

Want to get involved in developing SMF, then why not lend a hand on our github!

Main Menu

[4279]Internal Server Error on attachment upload

Started by ostinelli, July 09, 2009, 05:50:31 AM

Previous topic - Next topic

ostinelli

dear smf developers,

i have experienced something quite unpleasant that may happen when uploading attachments to a forum: you may be confronted with a 500 server error when doing so.

after careful digging i found the problem, how to reproduce it, and a suggestion on how to solve it.

simply put, if the attachments directory contains quite a considerable number of attachments [i have something close to 12,000], the scripts that loop into this directory while adding a new post go in timeout. yes, i could increase max_execution_time, but mine is currently at 120 and quite frankly i believe 2 minutes for a post is just too long a wait for a user.

the lines of code that i'm referring to are:


// Make sure the directory isn't full.
$dirSize = 0;
$dir = @opendir($modSettings['attachmentUploadDir']) or fatal_lang_error('smf115b');
while ($file = readdir($dir))
{
if (substr($file, 0, -1) == '.')
continue;
if (preg_match('~^post_tmp_\d+_\d+$~', $file) != 0)
{
// Temp file is more than 5 hours old!
if (filemtime($modSettings['attachmentUploadDir'] . '/' . $file) < time() - 18000)
@unlink($modSettings['attachmentUploadDir'] . '/' . $file);
continue;
}
$dirSize += filesize($modSettings['attachmentUploadDir'] . '/' . $file);
}
closedir($dir);


there are similar lines of code in 3 files [a simple search on '@opendir' will show them up easily]:

/Sources/Subs-Post.php
/Sources/Post.php
/Sources/ManageAttachments.php

these loops are used to:

- evaluate the dimension of the attachments directory;
- delete the temporary attachment files.

my suggestion to solve this issue is:

- have a DISTINCT directory where to upload temporary files, so that these can be deleted without looping 12,000 files with a preg_match just to find them [or use a specific suffix, so that opendir only loops these];
- save into a database setting the dimension of the attachments directory, so that it becomes unnecessary to loop EVERY TIME through the whole directory just to have this figure [or use an SQL query on the attachments table to sum up all the attachment sizes].

my only solution right now is simply to comment out all these lines in the three files and just live with the fact that i:

- cannot limit the attachments directory size;
- may end up with temp files which are never used.

i've seen that up to the newest RC this is still the same. i would advise to consider my suggestion, then it is up to you guys :)

cheers,

r.

karlbenson

Thanks for the report.

I'll need to speak to the devs about potential solutions to the problem that you have outlined.

12,000 attachments in one folder is probably a pain for most OS's.  In 2.0 RC1-1 we do have multiple attachment folders which might help you in the shortterm.

ostinelli

let me know if i can be of any help.

unfortunately i will not upgrade to 2 anyway. i have had to make significant patches to the original code to provide the integration i needed for my website. i wish it had multiple callbacks where to inject own code, but unfortunately the way smf is built doesn't allow me to do so, thus to upgrade i'd have to seriously rework through all the sources and template code.

keep it up, cheers,

r.

Advertisement: