Your attachment has failed security checks and cannot be uploaded

[badon]

I didn't see any drafts mods in my screenshot here:

Re: Your attachment has failed security checks and cannot be uploaded

Maybe you can see something there that might be the culprit?

[Illori]

the files will attach to the draft message successfully.

SMF has no feature to attach to a draft message, can you explain what you mean there?

[badon]

I mean that attempting to post without any text will upload and attached the files to the message, but will also give an error and ask for post text before it will accept a finished post. You can see that in my screenshot, here:

http://www.simplemachines.org/community/index.php?action=dlattach;topic=544243.0;attach=251864

In this post:

Re: Your attachment has failed security checks and cannot be uploaded

[nend]

The system will always do some sort of security checking no matter if the "Perform extensive security checks" box is checked or not.

On the false positive you got with the successful file upload. What you're seeing there is place markers after submitting your post, the script immediately returns and tells you that you have no message body. Even if files were uploaded they were never accepted or checked and this is why when you put your message in and then post you get the security error.

I did spend some time on this, tracing the functions and routine and couldn't find anything wrong with the code. As Suki pointed out on a previous reply, there were some malicious looking content at the beginning of the files that were being uploaded.

I would recommend reexamining your desktop and server security as this may be the cause. If any other forum users are having issues then I would also recommend reviewing linked and downloadable content from the server to make sure that none of these files aren't infected with anything nasty.

In the end if you want to bypass the security then you would have to make checkImageContents() in Subs-Graphics.php always return true. This is a pretty straight forward thing to do so I'll not go into details on how to achieve this as it is breaking security and shouldn't be encouraged.

[badon]

Thank you for your research, nend. Suki and Illori too.

Everyone who uploads files to our forum has this problem occasionally. If I get the file directly from them and try to upload it myself, which is exactly what happened in this case, I will get the same problem, so that rules out the files being tampered-with on the server side. I also experience this problem myself, and the odds of it happening increases if more and larger files are attached. As far as I can tell, this issue is completely random, with SMF tripping over random data combinations it doesn't like.

Based on your study of this problem, nend, it appears I was mistaken that it is possible to workaround the issue by not inputting text at the time of upload. In reality, the security checks are being done only after text is entered. That fits with my experimentation.

I altered checkImageContents() in Subs-Graphics.php to always return true, so there will never be any security checks on image files. I tested the result, and a known problem file was successfully uploaded:

Test of elimination of image file security check incorrect rejections

This problem killed our forum during the year it took to find a way to gut the erroneous security checks. Security doesn't matter very much on a dead forum. I'm not sure why some file were reject and some not rejected. It was so random I couldn't narrow it down to a single camera, application, editor, browser, etc, but with more effort I might be able to. Is there any reason to think these files are definitely malicious? If not, then is there any good reason to keep the code arbitrarily handing out random rejections?

We have probably had a thousand rejections, and if our forum didn't die because of them, we would have had far more. I am extremely skeptical that rejecting all those files serves any legitimate purpose, and I'm completely convinced not even 1 of them is actually malicious. The only thing worse than no security at all is constant alarms that everyone eventually ignores. Maybe a warning or notification for a manual admin check would be better than an unbeatable, blind rejection that cannot be turned off, cannot be overridden, and does nothing other than discourage people from using the forum.

[badon]

I just compared hashes from the server-hosted file and the offline file copy, and the hashes match, so that rules out server-side tampering. I'm surprised more people aren't having this problem. Maybe something else server-dependent could be increasing the likelihood of this happening?

[shawnb61]

I have definitely seen this as well.  The same photos fail an upload in the 2.1 beta as well.   

It all boils down to the edits in checkImageContents.  It would be interesting to know exactly which edits are failing in that lengthy preg_match.  I suspect the checks in there need to be rethought. 

[nend]

Looked at a few images you linked to and compared the rules for the non-extensive security checks.

One was due to following in code.

Code Select Expand

57 53 07

The other was due to following in exif information.

Code Select Expand

html
Which looks to be the page to where the image was downloaded from.

Two different rules triggering two different files.

[badon]

Both false positives, unilaterally rejected with no recourse, no intelligent examination, nothing = frustrated users = dead forum.

[landyvlad]

I am continuing to have this same problem with false positives and yes, it is extremely frustrating to forum members (and me of course) and drives some away in frustration which is very bad....

Like badon experienced, the instances it happens appear relatively random and can't be isolated to any one factor in particular.

I share badon's frustrations and opinions entirely.

The rules for non-extensive security need to be re-examined, as far too many are failing these checks which can't (aside from disabling checking entirely) be circumvented.

There is evidently a major issue which needs to be addressed PLEASE.

[badon]

I have been sending out instructions for manually disabling this problem in the code, to people who PM me to request it. It ought to be posted publicly. Is that OK?

[nend]

I have been sending out instructions for manually disabling this problem in the code, to people who PM me to request it. It ought to be posted publicly. Is that OK?

I would rather the operator enable re-encoding of potentially dangerous files as it wouldn't pose a possible security risk.

In short, no, it shouldn't be encouraged that forum operators break important security measures put in place to protect themselves and their members.

[landyvlad]

I'm trying the re-encoding thing - will see how that goes. Cheers to both of you

[badon]

We can't re-encode on our forum because we're trying to preserve quality images, and re-encoding degrades quality. Maybe re-encoding could be an option for use on manually confirmed malicious files.

As it is, I'm skeptical that a malicious JPEG could be exploited. It would probably require a permissions change or other higher-level access to the server, and at that point, the server has already been compromised. I doubt there's any way to exploit the users with an image file. Those kinds of exploits seem to have all been resolved after the 1990's - at least I don't recall any since then, largely because of the greater diversity of and higher quality of the software that reads them (like web browsers).

[Illori]

there have been security issues with SMF related to images containing scripts in the past that is why we are hashed them when saving [the name] and check for security in general on the images.

[shawnb61]

We can't re-encode on our forum because we're trying to preserve quality images, and re-encoding degrades quality. Maybe re-encoding could be an option for use on manually confirmed malicious files.

As it is, I'm skeptical that a malicious JPEG could be exploited. It would probably require a permissions change or other higher-level access to the server, and at that point, the server has already been compromised. I doubt there's any way to exploit the users with an image file. Those kinds of exploits seem to have all been resolved after the 1990's - at least I don't recall any since then, largely because of the greater diversity of and higher quality of the software that reads them (like web browsers).

I'm curious - what are the sources of your files?   I'm a bit of a photo nut and some of my .jpgs are being denied by SMF.  In my case they're taken on my Nikon D800 as NEF but exported as .jpgs after processing in Lightroom.  I am seeing more rejections, & have been wondering if the latest version of Lightroom is putting more content in the EXIF info that confuses the SMF security checker.

The reason you haven't seen so many attacks is because websites have gotten better at protecting themselves.  Exploits thru images are a very real - and current - threat:
https://www.trustwave.com/Resources/SpiderLabs-Blog/Hiding-Webshell-Backdoor-Code-in-Image-Files/ 

For the most strict security, re-encoding is probably the way to go because it is the ONLY way to remove the non-photo content.  I'm sure there are ways to re-encode without losing image quantity - copy the parts you want (all the pixels) and don't copy the other parts (exif, etc.). 

I suspect all the 'smart' photo features these days are causing false-positives in SMF's security check, which may need to adapt. 

[Illori]

if you open the image that is rejected in a text editor can you post the results in the next reply to this topic?

[shawnb61]

My files are all too big.   Each file that fails seems to fail on a different edit, even on the 'normal' edits.   I can copy excerpts of the files that fail below. 

Nothing for: iframe, eval, script\W

html:
s$«5c#ªÌV%¾—¸–ƒb5ÒRE¬f¿bÖ´   ®ý`‹éÓ·üË]r˜áqy›q/¢ÖN¡wI´ž±Z'Û?>ªÚ¬§N›DoÀájEÇLú-¬)ºyê#†'x[r·ô3sF†³%¡";ïý³öqx\ŒæÓL—1ÄO@abÏÖõÑeôðY3KÌ,kfMµm}æZœâ]V%©‰Âaê$ú]È
OļìhTmL;]V¥C¯g>
ôÇÓWÝSoÒL˜†Ž*...MyõÑ{[Iÿ êM]ß󪏩,u ãÿ
À#sês£ñiSìjÇžçöU¦
2Ò@ègÔå.bÈâ³Z­~2¯–eÐP>÷ù—‡ºõð‰–ÞË"¬GWp¬J±L,ŠY"
@°,,Vz^Všx+
èˆq

body:
HGîŸ"ÞA(1W¨£a‹\:£Ë''ƒ¾¤A±ýԁ{x!DFáÕ='¨©o"ÙD½ç•HûNÈç×;¨EÈ:#{û bOdyƒ²[ßÔ#Ì="FÇšÞ¡Øz$dö@"9   £bÙÙÎÉ—ŽÈÞ;+¡ÿÓù×–"›Ñ]T•.ZtU°#`Víî&©ÙҐYjݽµ4mHgp™ŒvÛKodЧÊöH°uZ*ÒÚSB,#Êj¾»#jhڏ(vG"ûrŠ²šyB¸KÉ´õFÔÒíŸÉƒV¨¤Ñ¶o%§¢<–ôFÛL7ºhÛ/<†­{'·©Mß«´ éDzӶÐ[X)¡›õfž‰~¬ÎËVÔmMdýY½RýU½–ͨÛÙ4méÙ/ÔÛÙlÛhÛI£l_ª7²_©·²ÜÑJhÛêmì'Ñ7²ÞZ•&†Ô›Ù¨·²ß¶ÐZšÿ Ô€èÔZº;p'bqз²_¨it¶ ±7õ¤t
töå¨9¨³ý—P5-QÊÿ g,,¿ÙôºÛi0-AÈÿ g,—û=uöR6Ò9ðô¿Ùë³±"ÀƒþÏGû=v¶¤iäðâþ z%þÏ=kfSòÂyC )³Êîyv,Äò8_¨;,,~ WpÆ/,,y}SÈá
úƒ—wb[òxpÿ P(ýA˹å,,l   äðáþ¢RýEË»±XO'‡õ¿RweßòÀAŒ<?ú",GFîËÐyMè%½"Èóߪ;²_ª8/Cä·²F›£ÏþªîÊ'Jå輆"y


[CF]WS[\x01-\x0C] (the binary character is an ACK, displayed here as an -):
¸   T–v*   ÁC¿Ü¡`¢ÓIÛnQQvmEßr‡·Ø)V"@B–þ—ö
¶šµ'^à7Jvy`La/`4¦ §TSlÀÚÍÏàáèß.:"O‰84X]È   †
Ø#-I1FwsAMS`Àû€ç¥À|"7›à
»ÀCØ¢ªiD&ìP5`)>@n0Q.ö
Ǻ InPÒì é¾ )I4M¤ªx i¼ ÿ ˆ¨'Ü