SMF converts attachments from .jpg to .png

[rawonam]

Currently I've got SMF 2.0 RC3, but probably RC1 had this problem too.

I'm attaching a test file, to see if the same happens here.

[rawonam]

OK, that was easy. Save the attachment file on your computer, look at the size and type.
It's now a PNG, sized 58.3 KB (59746 bytes), whereas I attached cocacola.jpg 8.94 kB.

That's really annoying, people post .jpg pictures of 500k and users download 3Mb .png files....

[Arantor]

No, this only happens as of RC3 and is a known bug.

[rawonam]

No, this only happens as of RC3 and is a known bug.

Is there a patch?

[Arantor]

I'm not sure, to be honest. I believe Kays has been doing some work on it, but if there was an answer it wouldn't be an official patch, what'll happen is the team will publish the changes, as well as other bug fixes, in RC4.

So any patch before then will be unofficial.

[rawonam]

So any patch before then will be unofficial.

Better than nothing. I'll try to look at the code myself, if I find nothing.

[Kays]

Currently the option to convert to jpeg/png is in the avatar settings and is also applied to attached images. With the next release there should be a similar option for attached images.

Download and install my resize attached images mod. I have addressed this problem in it as well as a few other problems with attached images

http://custom.simplemachines.org/mods/index.php?mod=2206

[rawonam]

Currently the option to convert to jpeg/png is in the avatar settings and is also applied to attached images. With the next release there should be a similar option for attached images.

Aha, thanks for the hint, though it is not everything that is happening here. I disabled resizing avatars and the forum still recompresses every uploaded image (now it makes them a few percents smaller). I want to disable any modifying of uploaded images, I'd appreciate if you told me which function and in which file is doing that, so that I could modify it. I'll save me probably half an hour or so of digging the SMF code to find it myself.

Download and install my resize attached images mod. I have addressed this problem in it as well as a few other problems with attached images

http://custom.simplemachines.org/mods/index.php?mod=2206

Thanks, but I don't use any mods on principle. I'm afraid of not being able to update to next versions of SMF. If I really need something changed, I modify the code manually.

[Kays]

The mod might be easier since there shouldn't be a need to use it with the next release. Or just use it for the clues.

The thing is that there are two definite bugs. The first being that the location for the memory increase needs to be changed. The second is that the check for reencoding is too aggressive, resulting in too many false positives and subsequent reencoding of too many images.

The third problem is saving as .png which you can disable in the avatar settings.

[rawonam]

OK, I disabled the reencodeImage function (just return true on entry). That solves everything. Avatars are still resized (and converted to PNG if I want to), uploaded images are untouched.
What's the use of this function anyway?

[Arantor]

The function is designed to re-encode potentially dangerous attachments to a different format that should kick out anything that seemed suspicious.

[Kays]

Thank you Arantor.

The checking for it is probably the cause of the extra memory usage you were questioning in another thread on this.

[rawonam]

The function is designed to re-encode potentially dangerous attachments to a different format that should kick out anything that seemed suspicious.

But it recompresses images, creating an image with reduced quality, am I right?
What's the security risk with untouched images? Why can it be dangerous?

[Arantor]

Untouched images can smuggle malicious content. Most image forms actually allow for some textual content to be stored in them, which can be interpreted by some browsers incorrectly.

For instance, a GIF that contains some Javascript as a form of comment, will often by actually run by Internet Explorer, because it sniffs the file to identify what it should do with it. (I don't know if IE8 does or not, certainly older versions that are still widely in use do this)

[rawonam]

Untouched images can smuggle malicious content. Most image forms actually allow for some textual content to be stored in them, which can be interpreted by some browsers incorrectly.

For instance, a GIF that contains some Javascript as a form of comment, will often by actually run by Internet Explorer, because it sniffs the file to identify what it should do with it. (I don't know if IE8 does or not, certainly older versions that are still widely in use do this)

Sounds weird to me, I'll try to google for info on this matter.
How come that IE interprets the contents of MIME type image/jpg as JavaScript and executes it?

I don't have restrictions on file extensions, if someone wants to attach a text file, or even an html page, is this a security risk too? Can IE execute some code from a text file?
And in case I do check for allowed extensions, will every possible file format be checked for malicious content and rewritten?

I want to make sure: does reencodeImage really create an image with reduced quality or it only removes some additional information? I don't know how GD resizing works, just want to understand that I'm not mistaken.

[rawonam]

There is one more thing connected with this issue: when an image file is reencoded (whether to jpg or png) the size of a new attachment is recorded that of the original file, not the saved one. Therefore it always indicated "wrong sizes" and errors in attachment directory. I don't know if that's the case with avatars, I hope you know what's going on.

[Arantor]

How come that IE interprets the contents of MIME type image/jpg as JavaScript and executes it?

Because it's stupid and tries to guess the contents rather than heeding the file type. Then again, how else would it deal with people who rename .png to .gif and expect it to work (and yes, it DOES happen, more than you'd think too)

In the case of non image attachments, they are always served with a header of binary so no browser should directly be attempting to open them, so it's not really an issue.

GD's reencoding should simply be a case of opening the image and resaving it in a different format, and dropping anything that isn't explicitly part of the original image data itself. Note that it will also likely damage animated GIFs in the same way.

Regarding sizes, that's another symptom of the bug then...

[Kays]

There is one more thing connected with this issue: when an image file is reencoded (whether to jpg or png) the size of a new attachment is recorded that of the original file, not the saved one. Therefore it always indicated "wrong sizes" and errors in attachment directory. I don't know if that's the case with avatars, I hope you know what's going on.

Another bug/problem. There's no code in place to update the database if an image does get reencoded. I don't know if this one has been addressed yet.

[engr_dood]

If anyone is willing to post or email an unofficial fix for this, I'd be most grateful.  I saw Kays' mod, but converting all graphic file types to JPG is not what I want to do.  Our forum has a 200k limit for attachments, but if someone uploads a JPG file that's just under the limit, its conversion to PNG is about 1.5 MB or more.  Also, in the forum there's electrical schematics and graphs being posted, and because of their "cartoon-like" properties (lots of blocks of solid color), the lossless compression of PNG looks way better than JPG (which makes schematics and graphs look fuzzy) and often ends up being surprisingly small in size too.  I've been wondering why the forum has been so slow lately, and experiments of fixing up the file type of large attachments via FTP have confirmed this bug to be the cause.

[rawonam]

If anyone is willing to post or email an unofficial fix for this, I'd be most grateful.  I saw Kays' mod, but converting all graphic file types to JPG is not what I want to do.

I'm gonna work on it today, I hope I'll have something in a few hours.