Large images causing thread to go 500, auto resize problem

[crimpson]

I'm using 2.0.7
Images that are too large (don't know exact dimensions) will cause a 500 error on the topic and not allow it to load. This problem has to do with the auto resizing feature for displaying the thumbnail attachments. max size I can upload so far is 2800pxx2800px. any larger than that and it breaks the thread.

What is the max supported dimensions for the resizing feature? I can't seem to find that anywhere. I struggled with this for most of the day until i figured out what was happening.

[Kindred]

It depends on your server...

Also, you need to upgrade  to 2.0.11 ASAP. You are missing several security releases

[crimpson]

I will update, but don't want to break any of my addons...
How does it depend on my server? I was in contact with them (ipage) and they increases my php memory but couldn't seem to figure out what the problem was.

[br360]

Patching (going from 2.0.7 to 2.0.8, then to 2.0.9 etc.) shouldn't break any addons. Take a look at this link which will show you how to upgrade (by patching)- http://wiki.simplemachines.org/smf/Patching

2800x2800 is actually fairly large for images. Are you talking about images uploaded as attachments through your forum, or images uploaded through a photo hosting site first and then the image url is copy and pasted into a post on your forum?

[nend]

I'm using 2.0.7
Images that are too large (don't know exact dimensions) will cause a 500 error on the topic and not allow it to load. This problem has to do with the auto resizing feature for displaying the thumbnail attachments. max size I can upload so far is 2800pxx2800px. any larger than that and it breaks the thread.

What is the max supported dimensions for the resizing feature? I can't seem to find that anywhere. I struggled with this for most of the day until i figured out what was happening.

What is your current limit?

Here is a quick formula
(Height*Width*4)/1000000 = Required Memory in MB to resize.

At 2800x2800 you'll need a little over 32 MB of RAM. That's not including anything else going on in the server.

[crimpson]

Imaes uploaded directly through the forum. attachment size restriction is 50000kb, image is about 28000kb, 7073x4761 for an example, that would cause the topic to go 500. the attachment would show up in the browse attachments area under the admin, and the 500 error will go away if i turn off image resizing.

[nend]

Image file size has nothing to do with the processing. Usually when a image is resized the pixels have to be resampled. So take the image width x the height at 4 bytes per pixel, this is your memory footprint for a 32 bit image.

This is just a rough example though, bytes per pixel can vary depending on the bit depth.

[crimpson]

so... more php memory? can the SMF image resizing system handle images of this size and larger?

[nend]

Actually, I was wondering about this myself and started digging in the sources.

In resizeImageFile(), SMF tries to increase the memory limit to 90MB. I am not sure if this has any negative effects if the limit was higher to begin with.

However looking through that file there is no check on image dimensions, just the overall file size. This was one of the things I was curious about.

Will in the posting phase if a thumbnail doesn't exist it tries to create one, if it does successfully it returns true, if not then false. Not too server intensive here.

But in the display phase if the thumbnail doesn't exist still it tries to create one again, I think this is what is causing your problem. The display procedure is already intensive enough and trying to resize the attachments again here just eats all the memory.

I don't know if to consider this as a bug, if a user posted a big image it may not fail on posting but once on the display it will cause a out of memory condition if it is big enough. 

[Kindred]

In other words...  Use the right tool for the activity.
Attached images should not be as large as you seem to be using.
Use an image hosting site like photobucket or use a gallery software that is designed to handle extremely large images

[crimpson]

Ok thanks.
So the only way to keep other users from making threads go 500 is by limiting the attachment size? What's to keep other users from trying to upload images with large dimensions?
I'm not getting an error message like "Image too large" or anything liek that, which would be nice. You submit your reply for instance, and it uploads the attachment, but then the thread just goes 500 with no explanation. That's not very good. Is that a problem on my end?

[nend]

While that being true, I think this may be one of those edge case bugs.

In any matter it doesn't prevent anyone from uploading a image and taking a advantage of any site running on limited resources who has attachments enabled.

All a attacker will need to do is keep adjusting the image dimensions so they get it just right where it goes through the post process but will fail on the display process. By doing so they can take a topic offline.

I just wonder why it failed on posting.

Also why is SMF trying to create the thumbnail again in Display.php, if it failed the first time it definitely is going to fail the rest.

[nend]

Ok thanks.
So the only way to keep other users from making threads go 500 is by limiting the attachment size? What's to keep other users from trying to upload images with large dimensions?
I'm not getting an error message like "Image too large" or anything liek that, which would be nice. You submit your reply for instance, and it uploads the attachment, but then the thread just goes 500 with no explanation. That's not very good. Is that a problem on my end?

In a way it is, the server your on is on the low memory side for what your members want to use it for. However I don't expect you to control what your members do. You do have a few options, you can disable attachments all together, disable resizing of large images or just comment out some code in Display.php to prevent the topic from being inaccessible.

If you do comment out the code, I believe the attachment would be shown as a link instead.

Code Select Expand

// A proper thumb doesn't exist yet? Create one!
if (empty($attachment['id_thumb']) || $attachment['thumb_width'] > $modSettings['attachmentThumbWidth'] || $attachment['thumb_height'] > $modSettings['attachmentThumbHeight'] || ($attachment['thumb_width'] < $modSettings['attachmentThumbWidth'] && $attachment['thumb_height'] < $modSettings['attachmentThumbHeight']))
{
$filename = getAttachmentFilename($attachment['filename'], $attachment['id_attach'], $attachment['id_folder']);

require_once($sourcedir . '/Subs-Graphics.php');
if (createThumbnail($filename, $modSettings['attachmentThumbWidth'], $modSettings['attachmentThumbHeight']))
{
// So what folder are we putting this image in?
if (!empty($modSettings['currentAttachmentUploadDir']))
{
if (!is_array($modSettings['attachmentUploadDir']))
$modSettings['attachmentUploadDir'] = @unserialize($modSettings['attachmentUploadDir']);
$path = $modSettings['attachmentUploadDir'][$modSettings['currentAttachmentUploadDir']];
$id_folder_thumb = $modSettings['currentAttachmentUploadDir'];
}
else
{
$path = $modSettings['attachmentUploadDir'];
$id_folder_thumb = 1;
}

// Calculate the size of the created thumbnail.
$size = @getimagesize($filename . '_thumb');
list ($attachment['thumb_width'], $attachment['thumb_height']) = $size;
$thumb_size = filesize($filename . '_thumb');

// These are the only valid image types for SMF.
$validImageTypes = array(1 => 'gif', 2 => 'jpeg', 3 => 'png', 5 => 'psd', 6 => 'bmp', 7 => 'tiff', 8 => 'tiff', 9 => 'jpeg', 14 => 'iff');

// What about the extension?
$thumb_ext = isset($validImageTypes[$size[2]]) ? $validImageTypes[$size[2]] : '';

// Figure out the mime type.
if (!empty($size['mime']))
$thumb_mime = $size['mime'];
else
$thumb_mime = 'image/' . $thumb_ext;

$thumb_filename = $attachment['filename'] . '_thumb';
$thumb_hash = getAttachmentFilename($thumb_filename, false, null, true);

// Add this beauty to the database.
$smcFunc['db_insert']('',
'{db_prefix}attachments',
array('id_folder' => 'int', 'id_msg' => 'int', 'attachment_type' => 'int', 'filename' => 'string', 'file_hash' => 'string', 'size' => 'int', 'width' => 'int', 'height' => 'int', 'fileext' => 'string', 'mime_type' => 'string'),
array($id_folder_thumb, $id_msg, 3, $thumb_filename, $thumb_hash, (int) $thumb_size, (int) $attachment['thumb_width'], (int) $attachment['thumb_height'], $thumb_ext, $thumb_mime),
array('id_attach')
);
$old_id_thumb = $attachment['id_thumb'];
$attachment['id_thumb'] = $smcFunc['db_insert_id']('{db_prefix}attachments', 'id_attach');
if (!empty($attachment['id_thumb']))
{
$smcFunc['db_query']('', '
UPDATE {db_prefix}attachments
SET id_thumb = {int:id_thumb}
WHERE id_attach = {int:id_attach}',
array(
'id_thumb' => $attachment['id_thumb'],
'id_attach' => $attachment['id_attach'],
)
);

$thumb_realname = getAttachmentFilename($thumb_filename, $attachment['id_thumb'], $id_folder_thumb, false, $thumb_hash);
rename($filename . '_thumb', $thumb_realname);

// Do we need to remove an old thumbnail?
if (!empty($old_id_thumb))
{
require_once($sourcedir . '/ManageAttachments.php');
removeAttachments(array('id_attach' => $old_id_thumb), '', false, false);
}
}
}
}
This takes care of the edge case on the topic view.

While posting if the image is any bigger, than it will result in a internal server error and since attachments are processed before hand, the post shouldn't go through.

It may be just easier to disable these options. 

Display image attachments as pictures under post
Resize images when showing under posts
Save thumbnails as PNG

[crimpson]

I just tested this with a simple image, all white, 7073x4761 and only 387kb, it'll make any thread go 500 if it has image resizing for thumbnail attachments.

Edit: actually right now its not accepting the attachment and returning 500 without posting the reply and crashing the thread... troubleshooting

[nend]

I just tested this with a simple image, all white, 7073x4761 and only 387kb, it'll make any thread go 500 if it has image resizing for thumbnail attachments.

Colors and file size don't matter, dimensions and bit dept does. Images are compressed, they have to be decompressed to be processed. Like I said 4 bits per pixel for 32 bit. So you take the height * width * 4 bits per pixel and you get your total bytes to load the image for processing. So your looking at over 134 megabytes just to process the image. Now factor in server processes, 

[crimpson]

Gotcha.
At the moment it won't replicate the problem. Instead of crashing the entire thread by successfully uploading the the image, it's returning an "internal server error" with this address mysite/index.php?action=post2;start=0;board=34
Earlier it would upload the image but the resizing feature would cause the thread to crash. I double checked that and made sure the address was correct.
something's different, maybe server load. I haven't changed much.
If anything, I hope this helps someone else. I'll report any new findings.

[br360]

Ok thanks.
So the only way to keep other users from making threads go 500 is by limiting the attachment size? What's to keep other users from trying to upload images with large dimensions?
I'm not getting an error message like "Image too large" or anything liek that, which would be nice. You submit your reply for instance, and it uploads the attachment, but then the thread just goes 500 with no explanation. That's not very good. Is that a problem on my end?

This mod might be able to help with that. You can decrease the attachment size and members that try to upload pics bigger than that will get an error message letting them know that their image is too big- http://custom.simplemachines.org/mods/index.php?mod=3255

[Illori]

has been reported to us as a bug in the past http://www.simplemachines.org/community/index.php?topic=516980 http://www.simplemachines.org/community/index.php?topic=470495

http://www.simplemachines.org/community/index.php?topic=532364.0

[nend]

I'll create a mod that will give you a option to set a memory limit for resizing. At least this way the images will still be attached, but it will not try to generate a thumbnail if the demensions are too large.

I think this would be the preferred way to handle this problem.

[nend]

@ crimpson

I did do some testing and when SMF changes the memory limit in the graphics subs it limits the max resolution to 4726 x 4726 or equivalent. This is hard coded in the graphics subs, so it doesn't matter if your host ups your memory, you still can't use it because SMF lowers it in the resize function.

Give this a try, let me know if it works for you.

In Sources/Subs-Graphics.php
Find

Code Select Expand

// Gif? That might mean trouble if gif support is not available.

Add before

Code Select Expand

$limit = round((($sizes[0] * $sizes[1] * 4) / 1000000) + 10); // Figure out requirements with a little overhead
if ($limit > 1024)
return false; // Not even going to try 1 gigs
elseif ($limit > 90) {
if(ini_set('memory_limit', $limit.'M') === false)
return false; // Server rejected
if (ini_get('memory_limit') != $limit.'M')
return false; // Server rejected silently
}

Here we are leaving the 90MB limit SMF imposed but if it isn't sufficient enough we are going to up it all the way to 1 gigabyte if needed.